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A.  INTRODUCTION 


1.0  Objective 

The  objective  of  this  project  was  to  utilize  visual 
steady-state  evoked  potentials  (SSEP),  head  movements,  eye 
movements,  and  eye  blinks,  to  evaluate  pilots'  response  to  a 
helmet-mounted  display  (HMD)  system  devised  to  increase  the 
efficiency  of  information  input.  Success  in  this  effort  would 
suggest,  in  addition,  that  these  measures  could  be  applicable 
as  general  psychophysiological  measures  of  workload.  The 
critical  test  of  these  measures  was  to  take  place  in 
simulated  flights  with  Air  Force  personnel. 

With  respect  to  the  SSEP  portion  of  the  project,  some  prelim¬ 
inary  work  was  necessary,  first,  to  establish  the  phenomenon 
in  a  laboratory  setting,  second,  to  explore  variables  affec¬ 
ting  the  robustness  of  the  effect,  and  finally,  to  apply 
these  findings  to  determine  optimal  parameters  for  stimula¬ 
tion  under  flight  simulation  conditions.  The  preliminary 
experimentation  was  carried  out  in  the  Washington  University 
Behavior  Research  Laboratory  in  St.  Louis.  The  results  of  the 
preliminary  laboratory  work  are  summarized  in  Part  B,  Section 
2.0. 

2.0  Apparatus 

The  apparatus  for  the  preliminary  SSEP  investigations  was 
a  masked  oscilloscope  (in  earlier  phases)  and  a  light  box  (in 
later  phases).  These  are  described  in  Part  B,  Sections  2.1 
and  2.7.  The  simulator  and  its  environment,  described  in 
Appendix  A,  was  a  B-1  adaptation  of  a  domed  F-15  simulator 
located  at  the  McDonnell-Douglas  simulation  facility  in  St. 
Louis . 

3.0  Organization  of  Report 

The  substantive  material  in  this  report  is  divided  into  five 
sections : 

In  Section  B,  steady-state  evoked  responses  are  discussed. 
This  includes  a  description  if  the  preliminary  work,  the 
rationale  for  the  selection  of  stimulation  parameters,  the 
procedures  followed  for  the  simulator  runs,  and  the  analysis 
and  results  of  the  SSEP  data  from  these  runs. 

Section  C  contains  the  results  of  the  analysis  of  saccades 
and  blink  data  derived  from  the  simulator  runs. 

Section  D  contains  a  review  of  the  literature  on  head  and  eye 
movements . 

In  Section  E,  the  results  of  a  manual  analysis  of  head 
movements  during  the  simulation  runs  are  presented. 
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B.  STEADY-STATE  EVOKED  POTENTIALS  ("PHOTIC  DRIVING") 

1.0  Purpose  and  Rationale 

The  purpose  of  this  portion  of  the  project  is  to  determine 
the  feasibility  of  using  steady  state  evoked  potentials  (also 
referred  to  as  "SSEPs",  "photic  driving"  or,  simply, 
"driving")  of  the  brain  as  an  indicator  of  workload.  The 
rationale  is  based  on  the  use  of  probe  event-related 
potentials  by  Bauer,  Goldstein,  and  Stern  (1987)  and  SSEPs  by 
Papanicoulaou  ( Papanicoulaou  &  Johnstone,  1984)  to  index  the 
residual  processing  capacity  of  an  individual  involved  in  a 
task.  In  contrast  to  the  "secondary  task"  method  of  assessing 
workload  (e.g.,  Isreal,  Chesney,  Wickens,  &  Donchin,  1980), 
these  methods  are  designed  to  minimize,  if  not  eliminate,  the 
active  or  passive  participation  of  the  subject  in  the 
assessment  procedure. 

The  data  suggest  that  the  greater  an  individual's 
"involvement"  in  a  cognitive  task,  the  less  residual  capacity 
there  is  to  process  nontask  material.  Assuming  that  "involve¬ 
ment"  is  roughly  translatable  into  workload,  it  would  be 
predicted  that  the  photic  driving  response  to  task-irrelevant 
flickering  stimuli  would  decrease  as  a  function  of  workload. 
Although  the  measure  abstracted  from  the  SSEP  v;as  somewhat 
different  from  that  used  here,  Wilson  and  O'Donnell  (1986) 
took  an  essentially  similar  position  in  their  work.  In  the 
present  application,  the  response  of  a  pilot  to  light 
flashing  imperceptibly  would  be  expected  to  be  inversely 
related  to  the  degree  of  workload  imposed  upon  him  by  various 
flight  conditions.  These  flashes  were  to  be  delivered  by 
flickering  a  helmet  mounted  display  during  sim\ilated  flight. 

2.0  Summary  of  Laboratory  Work  Prior  to  Simulation  Runs 

2.1  In  our  initial  efforts  in  the  laboratory,  the  stimulus 
was  presented  on  an  oscilloscope  masked  so  that  a  square 
approximately  32  mm  per  side  was  exposed. 

The  raw  data  went  through  several  treatment  stages.  The 
spectral  analysis  output  presented  power  in  1  Hz  bins  as 
a  proportion  of  the  total  power  from  0  to  100  Hz.  This 
was  done  for  two  conditions,  one  a  "control"  and  one  a 
stimulation  condition.  In  the  second  stage,  the 
proportion  of  power  at  a  given  frequency  under 
stimulation  conditions  was  expressed  as  a  percent  change 
from  the  proportion  under  "control"  conditions.  In 
summary,  it  was  found  that: 

1)  there  was  evidence  of  a  photic  driving  response  in 
all  subjects  tested; 

2)  generally,  flickering  at  lower  frequencies  (12,  18, 
and  24  Hz)  produced  driving  of  greater  magnitude 
than  did  higher  frequencies  (40,  44,  48  Hz),  though 
there  was  still  substantial  activity  over  and  above 
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"control"  levels;  note  that  the  higher  frequencies 
were  above  the  fusion  threshold; 

3)  electrode  placement  was  not  a  significant  variable 
in  determining  driving.  That  is,  0^^  and  ©2  responded 
similarly  to  the  stimulation; 

4)  over  a  one  minute  period,  no  consistent  time  effect 
was  observed; 

5)  monocular  stimulation  produced  photic  driving  levels 
comparable  to  those  obtained  under  binocular 
conditions ; 

6)  harmonic  driving  (driving  at  frequencies  lower  than 
the  stimulating  frequencies)  was  observed. 

2.2  In  an  effort  to  reduce  the  number  of  stimulating 
frequencies  to  a  more  manageable  set  for  data  reduction 
purposes,  and  to  retain  the  possibility  of  observing 
harmonic  driving,  the  number  of  stimulating  frequencies 
was  reduced  to  four:  4,  8,  12,  and  24  Hz.  Results  were 

1)  Individual  differences  existed  with  respect  to  the 
optimal  frequency  but  the  optimal  level  was 
consistent  within  subjects  over  several  testing 
sessions . 

2)  There  was  temporal  variability  in  the  driving 
response  for  which  we  could  not  account.  That  is,  it 
disappeared  and  reappeared  in  sequential  samples. 

3)  Foveal  stimulation  was  superior  to  peripheral  stimu¬ 
lation  (20°  off  center)  in  producing  driving. 

4)  The  question  of  whether  a  midline  occipital  or  more 
lateral  electrode  location  would  be  more  effective 
in  accommodating  changes  in  fixation  due  to  scanning 
in  the  operational  setting  was  put  to  the  test. 
Analyses  of  the  0^  and  0^^  output  did  not  result  in 
any  significant  differences.  While  it  was  decided  to 
use  the  0^  location,  the  use  of  a  more  lateral 
electrode  placement  was  not  ruled  out. 

2.3  A  large  proportion  of  EEG  activity  occurs  at  frequencies 
below  20.5  Hz.  There  is  a  high  potential  for  this  activ¬ 
ity  to  vary  from  segment  to  segment  causing  a  fluctua¬ 
tion  in  total  power.  Since  activity  in  all  bins  is 
expressed  as  a  proportion  of  total  power,  this  would 
have  the  effect  of  producing  apparent  variation  in  power 
in  all  other  bins  due  to  an  unrelated  shift  in  ^ow 
frequency  power. 

Thus,  we  compared  ouput  of  analyses  which  included  power 
in  the  0  -  20.5  Hz  frequency  range  as  part  of  total 
power  with  those  that  did  not.  In  the  process,  the 
subject  was  flashed  with  a  50  Hz  stimulus  as  an  initial 
attempt  to  explore  what  happened  at  higher  frequencies. 
1)  There  was  no  apparent  difference  in  driving  indices 
based  on  the  two  measures  of  total  power;  neverthe¬ 
less,  it  was  decided  to  exclude  the  lower  frequen¬ 
cies  from  total  power  in  future  measures  to  protect 
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against  the  possibility  that  this  source  of  artifact 
might  intrude  under  operational  conditions  not  yet 
represented  in  the  experimental  paradigm. 

2)  There  was  evidence  for  driving  at  the  higher  (50  Hz) 
frequency,  though  it  was  more  variable  than  at  the 
lower  frequencies  used  earlier.  Further,  the  driving 
response  was  not  observable  in  the  first  10  s  sample 
for  this  subject,  a  potential  disadvantage. 

3)  No  systematic  effect  of  time  was  noted. 

2.4  The  effects  of  sinewave  and  squarewave  modulated  light 
were  compared. 

1)  24  Hz  squarewave  stimulation  produced  clear  driving. 

2)  There  was  suggestive  evidence  for  harmonic  driving. 

3)  Again,  no  obvious  time  effect  was  observed. 

2.5  A  prototypical  helmet  with  a  reticle  display  was 
provided  by  AAMRL .  The  driving  response  produced  by 
flickering  the  reticle  display  (squarewave  at  24  Hz)  was 
compared  to  the  response  produced  by  a  24  Hz,  squarewave 
modulated  light  using  a  CRT  display. 

1)  Both  modes  of  stimulation  produced  driving.  In 
fact,  the  CRT  display  appeared  to  produce  more 
variable  driving  than  the  reticle  display. 

2)  At  24  Hz,  the  subjects  perceived  the  CRT  display  as 
flickering;  this  same  frequency  was  perceived  as  a 
steady  light  by  the  same  subjects  when  viewed  via 
the  reticle  display.  Th: s  was  seen  as  a  distinct 
advantage  since  it  would  be  necessary  to  flicker  the 
Kaiser  helmet  display  at  some  (optimal)  frequency 
above  fusion  threshold.  And  since  we  have  seen  that 
the  driving  response  is  generally  greater  at  lower 
frequencies,  this  information  was  encouraging. 

2.6  Up  to  this  point  the  effect  of  time  had  not  been 
evaluated  formally.  In  order  to  do  so,  one  subject  was 
stimulated  (at  24  Hz),  using  the  reticle  display,  for 
three  4  minute  blocks  separated  by  30  s  periods  of 
nonstimulation.  (EEC  from  the  nonstimulation  periods 
were  pooled  and  used  as  the  "control"  condition.) 

1)  Driving  appeared  in  the  initial  time  sample  but 
rapidly  deteriorated  suggesting  an  habituation  or 
fatigue  effect. 

2)  Following  each  30  s  nonstiraulation  period,  driving 
appeared  to  return,  but,  again,  only  briefly, 
reinforcing  the  suggestion  made  above. 

2.7  This  was  the  first  time  that  a  systematic  time  effect 
had  been  observed.  To  determine  if  this  effect  was  due 
to  stimulus  intensity,  a  more  intense  stimulus  was 
used.  A  light  box,  a  cube,  14  in  on  each  side  with  nine 
7.5  W  lamps  evenly  spaced  in  a  3  X  3  configuration  on 
the  insid"-  back  wall,  was  used  to  stimulate  a  subject  at 
24  Hz.  The  front  of  the  box,  a  milky  plastic  sheet,  was 
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one  foot  from  the  subject's  face.  The  illuminated  sheet 
subtended  an  angle  of  60°. 

1)  There  was  substantial  driving  across  two  2  min 
stimulation  periods  with  the  exception  of  two  unex¬ 
plained  10  s  chunks  near  the  end  of  the  second  run. 

2)  When  the  number  of  lamps  in  the  light  box  was 
reduced  from  nine  to  one,  no  clear  time  effect  was 
seen.  The  subject  in  this  condition  was  stimulated 
at  24,  30,  and  40  Hz.  At  24  and  40  Hz,  there 
appeared  to  be  an  increase  in  driving  over  the 
period  of  stimulation,  whereas  at  30  Hz  there  was  an 
initial  increase  followed  by  a  decrease.  At  all 
three  frequencies,  thus,  the  initial  measure  was 
low,  unlike  at  the  higher  intensity. 

2.8  Since  the  angle  subtended  by  the  helmet  mounted  display 
was  to  be  12°,  the  effect  of  reducing  the  size  of  the 
field  of  stimulation  from  60°  to  12°  was  assessed  using 
the  same  mode  of  stimulation  (i.e.,  the  'light  box', 
with  one  7.5  W  lamp).  Further,  since  in  the  operational 
setting  pilots  would  be  scanning  their  environment,  two 
laterally  placed  electrodes  were  added  to  explore  the 
possibility  that  peripheral  stimulation  might  be  picked 
up  more  effectively  by  lateral  electrodes.  Two 
additional  trials  were  tacked  on  to  the  end  of  this 
procedure  in  which  the  effect  of  rapid  blinking  was 
studied  using  large  (60°)  field  stimulation. 

1)  Initial  large  field  stimulation  produced  substantial 
driving  at  all  electrode  sites. 

2)  Reducing  the  size  of  the  screen  reduced  the  driving 
response  by  more  than  one  half.  Nonetheless,  the 
response  was  still  robust. 

3)  The  spread  of  the  driving  effect  to  lateral  elec¬ 
trodes,  very  strong  in  response  to  full  field 
stimulation,  was  minimal  in  response  to  12° 


stimulation.  Lateral  electrodes  do  not  appear  to  be 
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3.0  Rationale  for  Selection  of  Parameters  and  Procedures 

3.1  Frequency .  The  selection  of  stimulation  frequencies  was 
constrained  by  several  requirements.  Most  important  was 
that  the  subject  would  not  be  able  to  perceive  flicker 
and,  therefore,  that  the  frequencies  would  have  to  be 
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above  the  fusion  threshold.  A  second  reason  for  estab¬ 
lishing  the  fusion  threshold  is  the  suggestive  evidence 
in  the  literature  (discussed  in  Sec  3.2)  that  optimal 
driving  frequency  is  correlated  with  fusion  threshold. 
Accordingly,  choice  of  frequency,  ideally,  should  be 
related  to  the  fusion  threshold,  which  differs  across 
individuals.  Since  our  accessibility  to  the  pilots  was 
limited  to  simulation  time,  it  was  necessary  to  establish 
the  fusion  threshold  rapidly  during  that  period.  Time 
limitations  in  the  simulator  also  precluded  determination 
of  individual  optimal  driving  frequencies  in  any  direct 
way.  Consequently,  selection  of  frequencies  had  to  be 
made  in  a  rather  informal  manner.  That  there  were  four 
flights  (designated  alpha,  beta,  Charlie,  and  delta)  in 
each  of  the  four  conditions  (the  conditions  were  HMD  or 
noHMD,  coupled  with  Ingress  or  Refuel)  provided  an  ^  hoc 
rationale  for  selecting  four  stimulating  frequencies.  The 
procedure  for  doing  so  is  described  below. 

Another  parametric  decision  implicit  in  the  rationale 
presented  above  concerned  the  periods  for  which  the 
stimuli  were  to  be  presented.  Although  the  original 
intent  was  to  flicker  the  display  only  at  selected 
portions  of  the  flight,  several  factors  precluded  this 
procedure,  not  least  among  which  was  the  fact  that 
flicker  onset  produced  a  perceivable  drop  in  HMD  inten¬ 
sity.  For  another,  the  manner  in  which  a  particular  pilot 
flew  a  particular  flight  could  not  be  specified  a  priori 
so  there  was  no  guarantee  that  introducing  a  stimulus  at 
some  fixed  time  in  the  flight  plan  would  result  in  the 
same  functional  point  for  all  pilots.  Therefore,  the 
simplest  option  appeared  to  be  to  turn  the  flicker  on  at 
a  given  frequency  at  the  beginning  of  a  flight  and  off  at 
the  end. 

3.2  Intensity .  Early  work  is  abundantly  clear  in  indicating 
that  stimulus  intensity  is  positively  correlated  with 
fusion  threshold  (Hecht  &  Shlaer,  1936).  Other  data 
(Simonson  &  Blankstein,  1961)  suggest  a  correlation  of 
fusion  threshold  with  maximum  driving  frequency.  Unfor¬ 
tunately,  there  was  only  limited  control  that  could  be 
exercised  in  the  present  effort  over  this  potentially 
important  parameter.  HMD  intensity  was  under  the  pilot's 
control  via  an  uncalibrated  potentiometer.  The  pilots  had 
been  informally  instructed  to  turn  up  the  intensity  to  a 
level  that  was  "comfortable",  but  this  level  could  not  be 
measured  independently. 

3.3  Waveshape .  Although  our  early  lab  work  was  based  on  the 
assumption  that  the  waveshape  would  be  square,  the  actual 
state  of  affairs  was  more  complex  than  that,  and 
depended,  in  part,  on  the  time  it  took  to  repaint  the 
screen  in  each  pass.  In  a  spectral  analysis  of  the  HMD 
signal  picked  up  at  the  rear  cockpit,  it  was  found  that 
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the  percent  of  total  activity  found  at  the  nominal 
stimulating  frequency  was  never  greater  than  74%  and  was 
as  low  as  33%.  The  remainder  was  spread  across  the  spec¬ 
trum  with  consistent  secondary  peaks,  showing  as  much  as 
15%  power,  at  other  frequencies.  Further,  more  often  than 
not,  the  frequency  exhibiting  the  greatest  power  was  not 
the  nominal  stimulus  frequency  but  usually  1  Hz  higher. 

3.4  Control  for  Stimulus  Artifact .  In  laboratory  sessions 
preliminary  to  the  formal  flight  simulation  tests  with 
the  pilots,  subjects  underwent  flickering  with  eyes 
closed.  It  was  discovered  that  the  stimulation  induced  a 
nonphysiological  "coupling"  artifact  in  the  EEC  circuit. 
We  assumed  that  the  source  of  this  was  the  HMD  generator 
at  the  rear  of  the  helmet,  which  was  in  close  proximity 
to  the  EEC  electrode.  The  data  suggested,  nevertheless, 
that  response  power  at  the  critical  frequency  was  greater 
when  the  eyes  were  open  than  when  closed,  so  that  the 
contribution  of  the  brain's  processing  to  the  output 
could  still  be  evaluated.  This  required  the  inclusion  of 
an  eyes  closed  and  eyes  open  control  for  each  frequency 
used.  Since  the  artifact  would  be  expected  to  appear  in 
the  record  under  both  eyes  closed  and  eyes  open  condit¬ 
ions,  the  difference  in  the  EEG  record  under  these 
conditions  could  be  interpreted  as  activity  generated  in 
the  brain  in  response  to  the  flicker. 

4.0  General  Simulation  Procedure 

A  description  of  the  method  of  adaptation  of  the  F-15E 
cockpit  to  simulate  a  B-1  environment,  the  cockpit  displays, 
and  the  helmet  mounted  display  systems,  are  presented  in 
Appendix  A.  These  are  based  on  a  description  provided  by 
McDonnell -Douglas . 

4.1  Electrode  Application .  Seven  electrodes  were  attached  to 
each  pilot.  Four  electrodes  were  used  for  EOG  recording: 
one  each  above  and  below  the  right  eye  for  recording  of 
blinks  and  vertical  eye  movements,  and  one  each  lateral 
to  each  eye,  for  the  recording  of  horizontal  eye  move¬ 
ments.  For  EEG  recording,  one  electrode  was  applied  at  0 
and  one  to  ear  as  reference.  A  ground  electrode  on  the 
forehead  completed  the  array.  The  pilot  was  then  brought 
into  the  simulator  and  the  helmet  was  put  on. 

4.2  Calibration .  A  period  of  recording  followed  to  ensure  the 
integrity  of  the  recording  system.  To  calibrate  the  eye 
movement  system,  the  pilot  was  asked  to  fixate 
calibration  points  located  on  the  dome  display  10°  and 
45°  laterally  in  each  direction,  and  10°  vertically  in 
each  direction. 

4.3  Stimulation  Frequency .  The  first  step  in  selecting 
frequencies  was  to  establish  the  fusion  threshold.  This 
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was  done  through  a  modified  up-and-down  procedure.  The 
HMD  was  flickered  at  a  rate  determined,  a  priori ,  to  be 
below  any  pilot's  fusion  threshold.  The  pilot  was  asked 
to  report  whether  he  perceived  flicker.  If  so,  and  all 
did  at  this  level,  the  flicker  frequency  was  increased  in 
4  Hz  steps  until  he  reported  fusion.  At  this  point,  the 
procedure  was  reversed;  the  frequency  was  lowered,  halv¬ 
ing  the  step  interval  to  2  Hz,  until  he  again  reported 
flickering.  The  fusion  threshold  (cff)  was  defined  as  the 
average  of  these  reversal  points.  Of  the  four  frequencies 
for  each  pilot,  the  lowest  frequency  was  4  Hz  above  this 
average,  and  the  remaining  three  were  at  additional  4  Hz 
increments.  (The  one  exception  to  this  procedure  occurred 
when  one  pilot's  cff  was  so  high,  following  the  above 
formula,  that  the  highest  frequency  for  him  would  have 
been  within  range  of  the  60  Hz  filters.  The  range  of  his 
frequencies  was  compressed  so  that  the  highest  one  was 
not  above  54  Hz.)  These  points  were  recorded  on  a  form 
exhibited  in  Figure  1. 

This  overall  procedure  was  designed  to  sample  a  broad 
range  of  frequencies  and  seemed  the  best  of  our  options 
in  locating  the  optimal  frequency  for  each  subject. 

4.4  Orde r  of  Flight  Conditions .  As  seen  in  Figure  1,  a 
combination  of  two  variables  (HMD  vs.  noHMD,  and  Refuel 
vs.  Ingress  flights)  produced  four  general  conditions. 

The  order  of  the  conditions  for  each  pilot  differed, 
although  all  pilots  started  with  an  ingress  mission, 
either  HMD  or  noHMD,  and  continuing  with  ingress,  went 
from  HMD  to  noHMD  (or  reverse)  before  switching  to  the 
refuel  flights.  Thus,  subject  1  would  go  from  Ingress/HMD 
to  Ingress/noHMD  and  then  from  Refuel/noHMD  to 
Refuel/HMD.  For  Subject  2,  the  order  was:  ing/noHMD, 
Ing/HMD,  Ref/HMD,  Ref/noHMD.  Further,  for  each  of  these 
four  display-mode/mission-type  conditions,  there  were 
four  flights,  each  differing  somewhat  from  the  others  in 
that  condition.  The  flights  were  ordered  differently 
within  each  condition  and  were  run  with  only  enough 
interruption  to  set  up  the  next  flight  in  the  computer 
and  take  SWAT  ratings.  After  each  of  the  four  conditions, 
there  was  an  additional  brief  delay  to  change  recording 
tapes  and  run  a  nonflight  control  (described  in  Sec  4.5), 
when  that  was  called  for. 

4.5  Nonf 1 i ght  Stimulation  Control .  An  eyes-closed  and  an 
eyes-open  control  condition  were  introduced  prior  to  and 
following  each  of  the  two  HMD  conditions  (Ingress  and 
Refueling.)  While  alert  and  looking  straight  at  the  HMD, 
each  subject  was  given  a  half  minute  of  flickering  with 
his  eyes  closed  followed  by  a  half  minute  of  eyes-open 
flickering.  The  frequency  used  in  each  test  was  that  used 
in  the  upcoming  flight  (if  the  flight  was  the  first  in 
the  condition),  or  in  the  immediately  prior  flight  (if 
the  flight  was  the  last  in  the  condition.)  In  this  way,  a 
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nonflight  control  sample  was  obtained  for  each  frequency. 
Time  restrictions  prohibited  including  such 
eyes-open/eyes-closed  controls  at  the  beginning  and  end 
of  each  flight. 

4.6  Data  Recording .  Eight  channels  of  data  were  tape 
recorded;  Vertical  EOG,  Horizontal  EOG ,  EEG  (0  ), 

Vertical  Head  movement,  Horizontal  Head  movement, 

Stimulus  artifact  (flicker).  Voice  communication  between 
controllers  and  pilot,  and  Time  code. 

5.0  Data  Reduction  (All  data  were  digitized  at  a  sampling 
interval  of  5  ms) 

5.1  Spectral  analysis.  Our  spectral  analysis  (FFT)  program 
operates  on  samples  10.24  s  in  duration,  and  was  set  to 
express  the  power  in  1  Hz  wide  bins  from  0  to  55.5  Hz.  In 
addition,  total  power  from  0  to  100  Hz  was  determined  for 
each  sample. 

5.2  Treatment  of  Spectral  Analysis  Output .  The  power  in  each 
frequency  bin  was  expressed,  in  earlier  work,  in  two 
ways:  as  a  log  change  of  power  compared  to  that  in  selec¬ 
ted  neighboring  bins,  and  as  a  percent  of  total  power 
(from  0  to  100  Hz).  The  latter  proved  to  be  reliable  and 
essentially  redundant  with  the  log  measure,  so  the  data 
reported  here  will  be  restricted  to  the  percent  measure. 
Further,  since  we  anticipated  the  possibility  of  consid¬ 
erable  variation  in  energy  at  low  frequencies  from 
condition  to  condition,  variation  that  we  did  not  want  to 
contribute  arti factually  to  our  percent  measure,  we 
redefined  total  power  to  exclude  that  below  20.5  Hz.  In 
summary,  then,  the  power  in  a  1  Hz  bin  was  expressed  as  a 
percent  of  all  activity  from  20.5  to  100  Hz. 

The  percent  values  described  above  were  carried  one  step 
further.  The  frequency  in  every  test  condition  was 
compared  with  that  in  an  appropriate  baseline  condition 
(the  base  condition  will  be  identified  in  every  case). 

The  baseline  percent  was  subtracted  from  the  test  per¬ 
cent.  To  identify  the  source  of  the  contributions  to  the 
difference  scores,  in  later  analyses,  however,  the 
percent  power  in  each  condition  was  presented  without 
subtracting  a  baseline. 

5.3  Data  Selection  Strategy.  For  each  of  the  four  pilots  for 
whom  we  had  a  readable  EEG  record,  an  evaluation  was  made 
as  to  whether  he  showed  evidence  of  photic  driving. 
Frequency  analysis  output  during  the  nonflight  eyes-open 
condition  was  compared  to  the  appropriate  baseline:  the 
frequency  pattern  during  the  nonflight  eyes-closed 
condition.  A  positive  va’ue  for  this  statistic  (test 
minus  base)  was  taken  to  indicate  that  the  brain  response 
to  the  stimulating  frequency  was  over  and  above  any 
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artifact  induced  in  the  electrode  circuit  by  the  stimulus 
source.  This  was  done  for  each  of  the  four  frequencies 
for  each  subject. 

The  criterion  established  for  further  analysis  for  any 
subject  was  that  the  increase,  from  base  to  stimulation 
conditions,  at  a  particular  stimulated  frequency,  had  to 
exceed,  by  some  acceptable  amount,  the  change  in  that 
same  frequency  when  it  was  the  unstimulated  frequency  in 
the  three  other  stimulation  tests.  Thus,  for  example,  if 
A,  B,  C,  and  D  frequencies  were  used  for  subject  1,  and 
we  were  evaluating  his  response  to  frequency  D,  his 
percent  of  total  power  at  D  under  eyes-closed  conditions 
would  be  subtracted  from  his  percent  at  D  in  the 
eyes-open  condition.  The  criterion  for  continued  analysis 
for  this  subject,  then,  would  be  that  this  difference 
would  have  to  exceed  the  similarly  derived  values  for 
frequency  D  when  frequencies  A,  B,  and  C  were  the 
stimulating  frequencies. 

If  the  above  criterion  was  met,  we  went  on  to  digitize 
and  spectrally  analyze  that  subject's  data  for  those  two 
flights,  one  ingress  and  one  refuel,  for  which  he  was 
flickered  at  that  frequency.  It  should  be  noted  that  in 
the  latter  analyses,  the  base  condition  was  changed  from 
the  eyes-closed  condition,  used  in  the  above  evaluation, 
to  the  eyes-open  condition.  Activity  at  a  given  frequency 
in  the  eyes-open  nonflight  condition  was  subtracted  from 
the  activity  at  that  frequency  during  the  flight  segment 
being  evaluated.  Thus,  a  non-zero  value  would  be  obtained 
only  if  simulated  flying  altered  activity  that  was 
produced  in  the  alert,  but  non-flight,  condition.  This 
treatment  assumes  that  the  non-flight  condition  is 
essentially  a  zero  workload  condition. 

6.0  Results 

6.1  Nonflight  Eyes- ope n/Eyes- closed  Compar i sons .  Figures  2 
to  5  depict  the  results  of  the  initial  analyses  of  the 
nonflight  diagnostic  control  for  the  four  subjects.  For 
each  subject  there  are  four  sets  of  four  frequencies, 
each  set  (a  'set'  comprises  the  points  in  a  column  above 
the  frequency  designation  on  the  abscissa)  corresponding 
to  one  of  the  stimulating  frequencies.  Within  each  set, 
the  darkened  symbol  indicates  the  value  for  the  stimula¬ 
tion  frequency.  The  other  three  points  in  that  set 
represent  the  change  at  the  remaining  frequencies  when 
stimulation  was  at  the  frequency  designated  in  the 
legend . 

The  activity  increase  as  a  function  of  stimulation  can  be 
determined  for  a  given  frequency,  by  comparing  output  at 
that  frequency,  when  it  was  the  stimulation  frequency 
(darkened  symbol),  to  the  output  when  it  was  not  the 
stimulation  frequency,  the  remaining  three  open  symbols 
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connected  to  that  darkened  symbol.  Accordingly,  it  can  be 
seen  in  Figure  2  that  for  subject  1,  the  criterion  for 
photic  driving  was  met  by  a  single  frequency:  54  Hz.  The 
filled  symbol  at  54  Hz  was  above  the  connected  open 
diamonds;  the  latter  represent  the  output  at  54  Hz  when 
the  stimulating  frequencies  were  42,  45,  and  50  Hz.  In  no 
other  case  was  the  criterion  met;  for  example,  for  45  Hz, 
the  percent  power  increase  at  45  Hz,  when  stimulation  was 
at  42  or  54  Hz  (cf.,  triangles,  first  and  fourth  sets), 
were  greater  than  the  increase  at  45  when  45  was  the 
stimulation  frequency  (filled  triangle,  second  set),  thus 
ruling  out  45  Hz  as  a  criterion  frequency. 

Figure  3  displays  the  data  for  Subject  2.  The  data  are 
unconvincing  in  demonstrating  evidence  of  driving.  The 
only  frequency  that  approached  the  criterion  was  43  Hz 
and  in  that  case  the  activity  at  43,  when  the  stimulating 
frequency  was  39  Hz,  was  somewhat  higher  than  that  at  43, 
when  43  was  the  stimulating  frequency. 

Neither  of  the  remaining  two  subjects  showed  evidence  of 
meeting  the  criterion,  as  can  be  seen  in  Figures  4  and  5. 

In  summary,  only  one  subject,  and  he  at  only  one  frequen¬ 
cy,  54  Hz,  met  our  criterion  for  follow-up  analysis.  The 
following  is  based  on  subject  1  only  and  only  at  this 
frequency . 

6.2  Subject  1:  Overview  of  Output  at  Selected  Fr equenc i e s .  It 
should  be  emphasized,  at  the  outset,  that  the  following 
observations  are  based  on  a  limited  number  of  samples 
from  only  one  subject.  The  observations  offered  should  be 
accepted  with  this  in  mind.  It  is  our  view  that  the 
strength  of  the  interpretations  lies  in  the  coherence  of 
the  various  patterns  exhibited  in  different  conditions. 

Prior  to  viewing  the  effects  of  events  within  a  flight  on 
the  driving  response,  we  present  an  overview,  in  Figure 
6,  of  the  effects  of  the  general  simulation  conditions  on 
output  at  the  four  frequencies  for  subject  1.  The  data 
for  each  condition  are  in  terms  of  the  power  at  a  given 
frequency  expressed  as  a  percent  of  total  power  ("total 
power"  is  defined  as  activity  above  20  Hz)  in  that 
condition.  Four  conditions  are  plotted:  the  first  two  are 
the  nonflight  eyes-closed  and  nonflight  eyes-open 
conditions,  both  with  the  flickering  HMD.  The  third  con¬ 
dition  is  the  average  of  the  first  half  minute  for  the 
ingress  and  refuel  HMD  delta  flights  (the  54  Hz  flights). 
The  last  point  represents  the  averages  for  the  same 
periods  in  the  noHMD  (non-flickering)  condition. 

It  will  be  recalled  that  the  differences  between  the 
first  two  conditions  have  already  been  presented  in 
Figure  2  as  the  data  displayed  above  the  54  Hz  point  on 
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the  abscissa.  Figure  6  illustrates  that  the  difference  in 
percent  of  total  power  at  the  stimulating  frequency, 
observed  in  Figure  2,  was  not  due  to  a  particularly  low 
initial  point  but  to  a  real  increase  when  the  eyes  were 
opened.  The  power  for  the  four  nonstimulated  frequencies, 
in  contrast,  either  decreased  or  remained  unaffected.  The 
change,  moving  from  the  second  to  the  third  point, 
represents  the  effect  of  increasing  the  workload  from 
minimal  (nonflight,  condition  2)  to  some  moderate  level 
at  flight  onset.  Here  we  see  an  increase  in  all  nonstim¬ 
ulated  frequencies  while  the  power  at  54  Hz  decreases. 
This  pattern  is  consistent  with  the  hypothesis  that  an 
increase  in  the  imposed  workload  produces  a  reduction  in 
the  capacity  of  the  brain  to  process  task-irrelevant 
information.  Why  the  remaining  frequencies  exhibited  an 
increase  in  power,  which,  by  contrast,  was  substantial, 
is  not  clear  at  this  point. 

Explanation  of  the  factors  determining  the  final  condi¬ 
tion  (NoHMD)  is  a  bit  more  complex.  In  this  phase,  the 
pilot  has  the  same  assigned  task  as  in  the  analogous  HMD 
flight  condition,  but  is  not  being  flickered,  and,  in 
fact,  does  not  have  the  HMD  available.  The  difference, 
therefore,  would  have  to  be  due  to  the  flickering  plus 
any  workload  difference.  Since  the  workload  level,  as 
minimal  as  it  might  have  been  in  condition  3,  was 
hypothesized  to  have  already  reduced  responsiveness  to 
the  flicker,  we  have  little  reason  to  expect  that 
condition  4  would  substantially  modify  that.  In  the  ideal 
case,  of  course,  we  would  hope  that  the  54  Hz  output  in 
condition  4  would  be  lower  than  that  in  condition  3.  That 
is,  the  workload  in  condition  3  was  judged  to  be  only 
minimal  and  therefore,  following  the  argument  above,  it 
would  be  predicted  that  the  subject  would  still  retain 
some  residual  capacity  to  process  the  flickering 
stimulus.  The  present  results  do  not  allow  much  room  for 
further  depression  of  output  in  condition  3  in  periods 
within  that  flight  (reported  below)  when  load  was  judged 
to  be  higher  than  at  flight  outset.  Nevertheless, 
activity  in  condition  4  was  almost  identical  to  that  in 
the  nonload  (first)  condition;  this  would  suggest  that  at 
the  outset  of  condition  4,  we  are  observing  a  normal  EEG 
pattern.  Since  the  54  Hz  output  in  condition  1  differed 
little  from  this  level,  it  would  appear  that  little,  if 
any,  coupling  artifact  was  produced  by  flickering  a  pilot 
whose  eyes  were  closed. 

The  points  plotted  in  condition  3,  Figure  6,  represented 
the  average  of  the  opening  segments  of  54  Hz  ingress  and 
refuel  flights.  We  turn  now  to  an  investigation  of  the 
events  occurring  within  these  flights. 
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6.3  Ingress  Flight  Delta . 


6.3.1  Flight  plan .  The  flight  plan  for  the  criterion 
ingress  flight  (flickered  at  54  Hz)  is  presented  in 
Figure  7.  Waypoints  are  indicated  on  the  plan  by 
circled  numbers.  The  flight  is  broken  up  into 
segments,  indicated  by  numbers  1  to  11,  that 
correspond  to  the  periods  subjected  to  spectral 
analysis . 

6.3.2  Wi thin  Flight  Workload  Estimates .  Since  there  were 
no  threats  introduced  in  this  flight,  the  course  of 
the  flight  was  rather  homogeneous  from  a  workload 
perspective.  We  could  speculate,  nevertheless,  that 
in  the  initial  periods  (#1  and  #2)  the  workload  is 
relatively  low.  Though  the  period  just  before 
waypoint  2  might  have  usefulness  as  a  higher  workload 
segment,  since  a  turn  was  required,  the  EEG  record 
for  that  period  was  contaminated  by  artifact, 
precluding  analysis  of  this  data  segment. 

Speculating  further  about  workload  variation  within 
the  flight,  it  would  not  be  unreasonable  to  suppose 
that  workload  would  be  higher  during  the  turn  exe¬ 
cuted  between  waypoints  2  and  3  than  in  the  straight¬ 
away  following  waypoint  3.  These  estimates  are 
summarized  in  Table  1.  Note  that  the  workload 
category  descriptors  are  chosen  to  be  consistent  with 
those  in  Table  2  below. 

Table  1 

Workload  Estimates  for  Segments  of  Ingress 
Flight  Delta  for  Subject  1 

Workload  Estimate  Flight  Segment 
Low  1 ,  2 

Low  +  7,8,9,10,11 

Medium  3, 4, 5, 6 

As  can  be  seen  in  Figure  8,  there  does  not  appear  to 
be  any  relationship  between  the  estimated  workload 
level  and  modulation  of  the  power  at  54  Hz,  or,  for 
that  matter,  at  any  of  the  four  frequencies. 

6.4  Refuel  Flight  Delta 

6.4.1  Flight  Plan .  The  flight  plan  for  refuel  flight 
delta  is  presented  in  Figure  9.  The  typical  refuel 
flight  routed  the  tanker  on  a  U-shaped  course,  the 
bomber  approaching  from  the  opposite  direction. 
Several  variations  of  this  pattern  were  introduced  to 
prevent  anticipation  of  upcoming  events.  In  the  case 
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of  flight  delta,  this  took  the  form  of  delaying  for 
one  mile  the  rollout  of  the  tanker,  which  therefore 
overshot  the  bomber  course.  As  can  be  seen  in  Figure 
9,  approximately  a  minute  after  tanker  rollout,  the 
bomber  (dotted  line)  instituted  a  right  course 
correction,  apparently  overshot  the  tanker,  made  a 
another  correction  and  brought  the  bomber  on  the 
tanker  course.  As  with  the  ingress  flight,  the  refuel 
flight  was  broken  up  into  segments  for  analysis. 

These  are  indicated  along  the  tanker  course  in  Figure 
9  by  the  numbers  1  to  8. 

6.4.2  Wi thin  Flight  Workload  Estimates .  In  Segment  1,  the 
tanker  was  proceeding  along  a  straight  course,  as  was 
the  bomber,  there  was  no  possibility  of  the  pilot 
sighting  it,  and  consequently  these  were  judged  to  be 
low  workload  periods.  Though  the  tanker  initiated  its 
turn  during  segments  2  and  3,  there  was  little  for 
the  bomber  pilot  to  do  but  monitor  the  tanker's 
course  while  maintaining  his  own  course,  again  a 
relatively  low  workload  condition.  Following  the 
tanker  rollout,  however,  the  series  of  course 
corrections  initiated  by  the  bomber  pilot  probably 
represents  a  relatively  high  workload  condition, 
especially  since  the  initial  maneuvers  were  not  very 
effective.  Segment  4  would  reflect  this.  Segments  5 
and  6  are  straight  flight  to  overtake  the  tanker;  we 
would  rate  these  periods  somewhere  between  the  low 
workload  initial  segments  and  the  final  segments,  7 
and  8,  which  required  somewhat  finer  adjustments  in 
anticipation  of  rendezvous.  These  estimates  are 
summarized  in  Table  2. 


Table  2 

Workload  Estimates  for  Segments  of  Refuel 
Flight  Delta  for  Subject  1 


Workload  Estimate 
Low 

Medium 

High 


Flight  Segment 
1,2,3 
5,6 
4,7,8 


The  data  for  the  refuel  flight  are  plotted  in  Figure 
10.  In  addition.  Figure  11  contains  a  plot  of  the 
data  averaged  according  to  the  workload 
categorization  above  .  Unlike  for  the  ingress  flight, 
there  is  fair  agreement  between  the  estimated 
workload  and  depression  of  activity  at  54  Hz.  There 
is  no  such  relationship  between  estimated  load  and 
power  for  any  of  the  nonstimulated  frequencies.  Of 
course,  the  workload  estimates  were  rather  informally 
arrived  at  and  are  subject,  therefore,  to  the 
variability  of  such  measures.  However,  they  were 
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determined  by  three  independent  raters  whose 
judgments  were  almost  identical. 

6.5  SWAT  Ratings .  SWAT  ratings  were  taken  at  the  end  of  each 
flight,  and  are  summarized  in  Table  2A.  For  exploratory 
purposes,  paired  comparisons  were  carried  out  between 
relevant  conditions.  Only  one  condition  stood  out: 
stress,  under  Ingress-Threat  was  significantly  higher 
than  stress  under  either  Ingress-noThreat  or  refuel.  No 
comparisons  between  parallel  HMD  and  NoHMD  conditions 
were  statistically  significant. 

It  is  our  contention  that  the  usefulness  of  these  ratings 
in  the  present  contex  is  limited.  Whereas  the  SWAT 
ratings  apply  to  an  entire  flight,  the  measures  developed 
here  are  designed  to  reflect  workload  over  brief  time 
periods.  Observation  of  the  application  of  these  scales 
suggested  to  us  that  a  task  punctuated  by  a  brief  period 
of  high  workload,  but  otherwise  uneventful,  will  likely 
yield  an  invalid  measure.  We  suggest  that  the  point  in 
the  flight  at  which  this  brief  high  load  period,  such  as 
a  threat,  is  introduced,  is  critical;  it  is  more  likely 
to  be  reflected  in  an  increased  SWAT  rating  if  it  is  late 
rather  than  early  in  the  flight.  That  is,  the  closer  it 
is  to  the  time  at  which  the  rating  is  to  be  given,  the 
greater  its  effect  will  be  on  the  rating. 

Note  also  the  likelihood  that  the  rating  under  such 
conditions  will  not  really  characterize  the  entire  flight 
but  will  be  weighted  by  this  brief  event  within  the 
flight.  It  seems  to  us  that  SWAT  ratings  are  more 
applicable  in  a  situation  where  the  workload  is  relative¬ 
ly  uniform  throughout.  These  speculations  are  amenable  to 
experimental  test. 

7.0  Conclusions 

7.1  Simulator  Results.  Although  prior  laboratory  work  was 
encouraging  in  suggesting  that  steady-state  potentials 
(photic  driving)  could  be  established  with  stimulation 
frequencies  above  the  flicker  fusion  threshold,  the 
simulation  test  of  those  conclusions  cannot  be  said  to 
have  been  eminently  successful. 

Only  one  of  the  four  pilots  for  whom  we  had  acceptable 
EEC  records  demonstrated  what  we  considered  to  be 
evidence  of  photic  driving,  and  this  was  assessed  under 
the  most  ideal  circumstances  to  be  found  in  the 
simulation  setting:  eyes-open,  looking  directly  at  the 
flickering  HMD.  This  pilot  exhibited  an  increase  in  power 
at  54  Hz  when  it  was  the  stimulating  frequency  but  no 
change  at  54  Hz  when  stimulated  at  one  of  the  other  three 
frequencies . 
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TABLE  2A 


SWAT  RATINGS  AS  A  FUNCTION  OF  THREAT, 
FLIGHT  TYPE,  AND  USE  OF  HMD 


INGRESS-NO  THREAT 
HMD  NoHMD 


s 

TIME 

EFFORT 

STRESS 

TIME 

EFFORT 

STRESS 

1 

2.0 

2.0 

2.25 

2.0 

2.0 

2 . 5 

2 

1.3 

1.67 

2.0 

1.3 

1 .25 

1 . 3 

3 

1.3 

2.0 

1.0 

1.3 

2.0 

1.0 

4 

1.0 

1.0 

1.0 

1.5 

2.0 

1 . 0 

5 

1.5 

2.0 

1.5 

1.0 

2.0 

1.0 

M 

1.42 

1.73 

1.55 

1.42 

1.85 

1.36 

S 

TIME 

HMD 

EFFORT 

INGRESS 

STRESS 

-THREAT 

TIME 

NoHMD 

EFFORT 

STRESS 

1 

2 

2.0 

3.0 

3.0 

1.0 

2.0 

1.0 

3 

2.0 

2.0 

2.0 

2.0 

2.0 

2.0 

4 

1.0 

1.5 

2.0 

2.0 

2.0 

2.0 

5 

2.0 

2.5 

2.0 

2.0 

2.0 

1.0 

M 

1.75 

2.25 

2.25 

1.75 

2.0 

1.5 

S 

TIME 

HMD 

EFFORT 

REFUEL 

STRESS 

TIME 

NoHMD 

EFFORT 

STRESS 

1 

1.0 

1 . 5 

2.0 

1.5 

2.0 

2.25 

2 

1.0 

1.5 

1.5 

1.0 

2.0 

2.0 

3 

1.0 

2.0 

1.0 

1 . 5 

2.0 

1.0 

4 

1.5 

2 . 0 

1.25 

1 . 3 

2.0 

1.7 

5 

1.0 

2.0 

1.5 

1.0 

2.0 

1.5 

M 

1.1 

1.8 

1.45 

1.26 

2.0 

1.69 

AVERAGE  OF  ALL 
HMD 

CONDITIONS 

NoHMD 

TIME 

EFFORT 

STRESS 

TIME 

EFFORT 

STRESS 

1  .  42 

1.93 

1.75 

1  .  48 

1.95 

1 . 52 
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Although  no  definitive  statement  can  be  made  concerning 
why  other  subjects  did  not  exhibit  evidence  of  driving, 
there  are  many  factors  that  could  be  pointed  to  as 
potentially  significant  contributors.  Preliminary  work 
clearly  demonstrated  substantial  individual  differences 
in  optimal  frequency  for  photic  driving.  In  the 
simulation  procedure,  there  was  no  time  available  to 
ascertain  this  value  for  any  pilot;  the  stimulating 
frequencies  were  selected  in  a  manner  that  could  have 
missed  the  optimal  for  a  given  pilot. 

A  second  potential  source  of  the  difficulty  in  demonstra¬ 
ting  photic  driving  more  effectively  is  the  low  intensity 
of  the  HMD.  As  indicated  at  the  outset  of  the  EEG 
section,  the  intensity  was  informally  set  by  the  pilot 
to  his  maximum  comfort  level  and  not  cal ibratable .  While 
the  system  was  capable  of  higher  intensities,  the  pilots 
preferred  the  dimmer  displays.  Yet  another  problem,  along 
these  same  lines,  is  the  power  output  at  the  stimulating 
frequency,  which  ranged  from  as  little  as  33%  to  a 
maximum  of  74%  of  total  power;  most  commonly,  the  output 
was  in  the  low  50s.  Thus,  the  intensity,  low  in  the  first 
place,  was  further  attenuated,  both  factors  reducing  the 
probability  of  the  driving  response. 

Another  possibility  was  that  pilots  may  have  interpreted 
scintillation  of  the  display,  which  is  a  property  of  the 
system,  as  flicker,  with  a  consequent  higher  estimated 
flicker  fusion  level.  This  would  increase  the  frequencies 
of  the  flicker  used  and  thus  also  tend  to  reduce  driving. 

Finally,  only  enough  time  was  available  to  run  two 
baseline  eyes-open/eyes-closed  tests  for  each  set  of  four 
flights,  one  at  the  beginning  and  one  at  the  end  of  each 
HMD  set.  This  did  not  insure  a  high  degree  of  reliability 
of  these  measures.  As  mentioned  earlier,  a  more  optimal 
procedure  would  have  been  to  run  a  series  of  eyes-open/- 
eyes-closed  trials  before  and  after  each  flight  at  the 
frequency  of  stimulation  of  that  flight. 

We  would  like  to  add  to  the  above  discussion,  that  the 
factors  mentioned  are  speculative.  It  is  quite  possible 
that  photic  driving  is  simply  not  demonstrable  in  the 
great  majority  of  subjects  under  these  conditions. 
Nevertheless,  in  the  absence  of  control  over  these 
potentially  significant  variables,  we  cannot  determine 
which  conclusion  is  correct. 

We  will  turn  now  to  the  workload/driving  relationship 
within  the  ingress  and  refuel  flights.  With  respect  to 
ingress,  the  data  do  not  offer  any  evidence  for  any 
relationship.  The  only  flight  data  that  seem  to  support 
the  hypothesis  that  driving  reflects  workload  comes  from 
the  refuel  flight.  It  is  not  possible,  however,  to 
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evaluate  the  magnitude  of  the  workload/driving  relation¬ 
ship  that  was  observed  in  that  flight.  Though  they  appear 
to  agree  well,  that  is,  increases  in  estimated  workload 
are  accompanied  by  average  decreases  in  the  EEG  response 
to  the  flicker,  the  individual  values  contributing  to 
this  average  do  not  sustain  this  view.  Thus,  we  see  in 
Figure  10  that  variation  among  the  first  three  54  H2 
points,  which  comprise  the  estimated  low  workload 
subtask,  is  greater  than  the  variation  in  percent 
activity  across  workload  levels  as  seen  in  Figure  11.  It 
is  tempting,  of  course,  to  speculate  that  the  beginning 
of  the  flight  (points  1  and  2)  represents  a  slightly 
higher  load  than  later  (point  3),  but  without  objective 
evidence  to  back  up  this  ^  hoc  speculation,  it  would  not 
be  profitable  to  argue  the  point. 

7.2  Overall  Evaluation  of  EEG  Effort.  Despite  the  weakness  of 
the  evidence  for  a  workload  effect,  several  points  have 
been  established  by  this  work.  First,  a  reasonable  EEG 
can  be  recorded  in  the  simulation  setting,  a  fact  not 
altogether  apparent  at  the  outset  of  this  research.  The 
anticipated  coupling  artifact,  induced  electrically 
(i.e.,  nonphysiologically )  by  the  flickering  stimulus  in 
earlier  trials,  appeared  to  have  been  controlled.  Second, 
flickering  the  HMD  can  be  accomplished  without  the 
subject's  awareness,  and  result  in  interpretable  EEG 
output.  Third,  the  percent  of  total  activity  seems  to  be 
a  sensitive  measure  of  the  following  response  of  the 
brain  to  flickering  light,  and  is  essentially  redundant 
with  the  log  measure  and  without  some  negative  features 
of  the  latter.  Finally,  there  seemed  to  be  a  pattern  in 
one  subject  that  was  consistent  with  a  driving  effect, 
and  also,  if  one  compares  the  driving  under  different 
conditions  (Figure  6),  evidence  of  a  workload  effect. 
However,  the  latter,  while  demonstrable  on  average,  was 
not  reliable  in  each  replication. 

It  is  clear  to  us  that  this  area  needs  more  controlled 
laboratory  research  before  the  feasibility  of  its 
application  can  be  adequately  tested.  The  following 
section  will  be  devoted  to  outlining  such  research. 

7 . 3  Projection  for  Future  Research 

7.3.1  As  is  apparent,  the  choice  of  stimulating  frequency 
for  a  given  subject  is  a  critical  one.  Not  only  are 
there  differences  among  individuals  in  the  frequency 
that  optimally  produces  photic  driving,  but  there  is 
reason  to  believe  that  intensity  is  a  significant 
variable  in  determining  this  level.  It  was  shown 
earlier  in  this  report  that  intensity  reduction 
decreased  driving  at  a  frequency  chosen  to  be 
optimal.  What  had  not  been  ascertained,  however,  was 
whether  at  the  lower  intensity  the  optimal  frequency 
had  changed. 


Several  lines  of  evidence  suggest  that  intensity  does 
have  a  bearing  on  the  optimal  frequency  for  driving. 
One  piece  of  evidence  for  this  inference  comes  from 
the  classical  literature  (Hecht  &  Shlaer,  1936), 
viz,,  that  critical  flicker  frequency  (cff),  the 
frequency  at  which  a  flickering  stimulus  is  perceived 
as  steady,  increases  as  a  function  of  intensity. 
Second,  Simonson  &  Blankstein,  (1961)  found  that 
optimal  driving  frequency  as  well  as  cff  increase  as 
a  function  of  age.  We  believe  that  these  reflect  the 
declining  capacity  of  the  aging  nervous  system  to 
resolve  rapidly  occurring  stimuli,  and  therefore, 
that  these  correlated  age  effects  are  both 
manifestations  of  the  same  mechanism.  It  follows  from 
this  and  the  cff /intensity  relation  that  intensity 
will  affect  optimal  driving  frequency  as  it  does  cff. 
Note  that  this  model  is  in  conflict  with  the  view 
implicit  in  Wilson  and  O'Donnell's  (1986)  report.  In 
their  work,  the  cognitive  significance  of  driving  in 
various  frequency  ranges  was  treated  as  if  these 
frequencies  were  constant  for  each  subject,  analogous 
to  "resonant"  frequencies  that  reflect  activity  in 
distinct  cognitive  systems.  The  argument  presented 
here  suggests  that  these  frequencies  vary  with 
stimulus  parameters  although  there  may  still  be 
non-overlapping  systems  subserving  distinct 
functions . 

This  inference  may  be  tested  by  flickering  subjects 
at  a  variety  of  frequencies  each  presented  in 
combination  with  a  series  of  intensities.  Each 
frequency/intensity  combination  would  be  repeated 
many  times.  Spectral  analyses  would  be  performed  and 
the  power  output  in  1  Hz  bins  would  be  calculated.  In 
all  cases,  the  stimulus  would  be  a  pure  sine  wave, 
which  produces  greater  (%  total)  power  than  a  square 
wave.  In  this  manner,  it  could  be  determined  whether 
optimal  frequency  shifts  as  intensity  changes. 

7.3.2  Following  the  above  investigation,  it  would  be 

essential  to  determine  whether  workload  affects  the 
driving  response.  The  simulator  is  not  the  ideal 
environment  to  demonstrate  this  phenomenon.  Several 
of  the  potentially  confounding  variables  have  already 
been  discussed,  and  we  may  add  variation  in  the 
subject's  direction  of  gaze  during  the  period  in 
which  an  EEG  sample  is  to  be  taken  for  spectral  anal¬ 
ysis.  Thus,  if  the  subject  shifts  his  gaze  during 
such  a  period,  so  that  the  stimulus  falls  on  the 
peripheral  retina,  our  data  show  that  this  will 
reduce  output  at  the  stimulating  frequency.  To  com¬ 
pound  matters,  one  can  conceive  of  a  correlation 
between  such  behavior  and  degree  of  workload.  That 
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is,  under  just  those  conditions  where  the  workload 
demands  are  increasing,  the  pilot  may  feel  it  necess¬ 
ary  to  scan  the  environment  to  acquire  information 
not  available  on  the  HMD.  This  would  have  the  effect 
of  reducing  the  driving  response  and  artifactually 
support  the  hypothesis.  It  is  obvious  that  this  will 
remain  a  problem  to  be  addressed  in  subsequent 
research.  This  study,  however,  will  concentrate  on 
demonstrating  the  driving/workload  relationship  under 
ideal  conditions;  factors  that  may  interfere  with  the 
relationship  and  effective  means  of  compensating  for 
them  will  be  deferred  to  a  later  date. 

Subjects  will  be  given  tasks  that  differ  in  workload. 
The  tasks  will  be  designed  so  that  each  is  relatively 
uniform  in  workload  throughout  its  course,  or  at 
least  that  there  are  sufficiently  long  periods  within 
a  task  that  are  at  a  specifiable  and  constant 
workload  to  permit  repeated  EEG  sampling  at  that 
level.  The  task  will  be  designed  so  that  the 
attention  to  the  flickering  visual  stimuli  is 
necessary  in  order  to  carry  out  the  task.  An  example 
of  such  a  task  is  that  used  by  Vicente,  Thornton,  and 
Moray  (1987),  in  which  the  subject  guicJed  a 
computer-simulated  "hovercraft"  down  a  winding  river. 
To  manipulate  workload,  they  varied  both  the  width  of 
the  river  and  the  standard  deviation  of  the 
turbulence  distribution  added  to  the  joystick  output. 
For  present  purposes,  the  rectangle  of  light  (the 
hovercraft)  on  the  computer  terminal  would  be 
flickered.  This  task  would  meet  the  criterion  of 
insuring  that  the  subject  is  attending  to  the 
flickering  stimulus. 

7.3.3  Another  direction  that  this  project  might  take 

concerns  the  changing  nature  of  the  task.  At  various 
points  during  a  flight,  the  character  of  the  task  may 
differ  considerably.  At  one  point,  the  pilot  may  be 
carefully  guiding  his  craft  through  a  series  of 
difficult  maneuvers.  At  another,  he  may  be  absorbing 
information  presented  to  him  either  visually  or  by 
audio  communication.  At  yet  another  phase  of  the 
flight,  the  pilot  may  be  integrating  information  and 
considering  what  the  best  course  of  action  would  be. 
These  activities  may  represent  qual i tatively 
different  workloads  for  the  pilot. 

The  question  for  us  is  whether  this  measure  of 
workload,  suppression  of  photic  driving,  is  equally 
sensitive  to  these  essentially  different  tasks:  Does 
it  simply  represent  a  general  measure  of  attention 
without  regard  to  the  nature  of  the  task?  Or,  does 
the  quality  of  the  task  play  a  vital  role  in  the 
activity  of  the  population  of  neurones  detected  by 
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our  recording  electrode ( s ) ?  Data  supporting  the 
latter  have  been  reported  by  Wilson  (1981).  If  this 
proves  to  be  the  case,  it  would  suggest  the  use  of 
multiple  recording  sites  to  assess  the  component 
processes  drawn  on  by  a  particular  task.  This 
assumes,  of  course,  that  there  exist  cortical  sites 
whose  activity  differentially  reflect  distinct  task 
dimensions . 
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C.  SACCADE  AND  BLINK  ANALYSIS 


1.0  Measurement  of  Blinks  and  Vertical  Saccades 

Saccade  and  blink  detection  algorithms  were  used  for  this 
analysis.  Since  current  display  routines  allow  us  to  view 
only  two  channels  of  data  concurrently,  the  analysis  was  a 
two-step  procedure.  We  first  analyzed  blinks  and  horizontal 
saccades  to  allow  us  to  evaluate  each  of  these  parameters 
independently  and  to  abstract  information  about  the 
co-occurence  of  blinks  and  horizontal  saccades.  The  second 
analysis  involved  blinks  and  vertical  saccades.  We  were 
concerned  here  as  well  with  the  temporal  r '' ationship  between 
blinks  and  vertical  saccades. 

One  problem  encountered  in  the  analysis  of  blink  data  was  the 
occurrence  of  many  blinks  in  coordination  with  vertical  eye 
movements.  Since  one  cannot  identify  vertical  saccades  made 
during  a  blink  with  the  use  of  the  electrooculographic  (or 
any  other)  procedures,  the  occurrence  of  such  saccades  had  to 
be  inferred.  Accordingly,  a  saccade  was  presumed  to  have 
occurred  if  the  vertical  position  of  the  eye  following  the 
blink  was  not  the  same  as  it  was  at  blink  initiation.  For 
example,  in  Figure  12A  the  voltage  level  at  termination  of 
the  blink  (and  saccade)  is  below  the  voltage  at  onset.  This 
indicates  that  the  eye  position  at  termination  was  higher 
than  it  was  at  blink  initiation.  We  have  not,  as  yet, 
developed  a  computer  algorithm  to  make  this  assessment;  it 
was  made  using  manual  editing  routines. 


ABC 
Figure  12 

Our  blink  identification  algorithm  requires  that  the  voltage 
following  the  trough  of  a  deflection  return  to  a  level  at 
least  half  the  distance  between  blink  initiation  and  maximum 
lid  closure.  If  it  does  not  do  so  in  a  specified  time  (200 
msec  in  the  present  analysis),  the  pattern  is  not  accepted  as 
a  blink.  Our  analysis,  thus,  underestimates  the  frequency  of 
blink  occurrence. 

A  second  problem  observed  in  the  present  data  set,  and  one 
seldom  encountered  by  us,  was  a  combination  of  eye  movements 
that  could  be  interpreted  (by  our  algorithm)  as  a  blink.  For 
example,  an  upward  saccade  follov/ed  by  a  downward  glissade 
(slower  eye  movement)  that  returned  the  eye  somewhat  more 


than  half  the  distance  of  the  initiating  saccade,  (see  Figure 
12B)  would  be  interpreted  as  a  blink  by  the  algorithm.  These 
eye  movements  could  be  discriminated  from  true  blinks  on  the 
basis  of  the  difference  in  the  form  of  the  final  voltage 
deflections:  the  glissade  terminating  the  blink  is  sharper 
than  the  glissade  portion  of  the  saccade  (compare  A  and  B, 
Figure  12^  These  pseudoblinks  (i.e.,  they  were  identified 
as  blinks  by  the  computer  program)  were  manually  edited  from 
the  data. 

We  are  somewhat  puzzled  by  the  glissade  portion  of  the  eye 
movement.  Though  these  eye  movements  appear  similar  to  those 
that  compensate  for  head  movements  in  the  vertical  plane,  we 
assured  ourselves  that  they  were  not  such  compensatory  eye 
movements  by  relating  them  to  the  head  movement  data  and 
finding  no  consistent  relationship. 

1.1  Data  Output;  Hard  Copy 

1.1.1  Blinks .  Table  3  depicts  "raw"  outputs  from  these 
analyses,  which  contain  information  about  each  blink 
and  saccade.  Column  1  identifies  the  time  of 
occurrence  of  either  a  blink  or  saccade.  Columns  2 
through  7  present  information  about  blinks  that  were 
initiated  at  the  time  specified  in  column  1.  (For  the 
letters  in  the  following  explanations,  refer  to 
Figure  12C.)  Column  2  identifies  the  time  between 
successive  blinks,  and  column  3,  blink  amplitude  (D 
minus  A,  in  A/D  units,  proportional  to  voltage). 
Column  4  ("half  duration")  contains  the  time  between 
the  point  where  a  blink  achieves  half  of  its  total 
amplitude,  and  the  point,  in  the  reopening  phase,  at 
which  it  recrosses  that  same  (voltage)  level  (F  minus 
B).  "Window  duration"  (column  5)  is  similar  to  the 
half  deration  measure  except  that  it  is  for  an 
optional  percentage  specified  by  the  operator.  That 
window,  labeled  "user-set  window"  in  the  table 
heading,  was  20  percent  (i.e.,  20%  up  from  full 
closure)  in  this  analysis  (E  minus  C).  Column  6, 
"descent  (50%)",  is  the  time  from  half  closure  to 
full  blink  closure  (D  minus  B),  while  column  7, 
"closing  duration",  refers  to  the  time  between  blink 
initiation  and  full  closure  (D  minus  A). 

1.1.2  Saccades .  Columns  labeled  from  A-E  in  Table  3  deal 
with  saccade  information.  Column  A  refers  to  the  time 
between  successive  saccades.  Column  B  refers  to 
saccade  duration,  i.e.,  the  time  between  saccade 
initiation  and  termination.  Column  C,  labeled 
"drift,"  refers  to  voltage  level  shifts  between 
successive  saccades,  i.e.,  the  change  in  position  of 
the  eye  from  the  termination  of  one  saccade  to  the 
initiation  of  the  next  one.  Such  shifts  may  reflect 
the  occurrence  of  slow  pursuit  movements  between 
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tabu:  3a 


COMBINED  BUNK  AND  SACCADE  DATA  FOR  FILE:  DKl : 5ANA2H . DAT 
?ine.v  BLINK  AMPLITUDE  CRITERION:  200  A/D  UNITS 

\)ert\ai\  -I-  BLINK  MAXIMUM  DURATION:  200  MSECS 

DROP  CRITERION:  100  A/D  UNITS 
^  USER-SET  WINDOW:  20  PERCENT 


SACCADE  MINIMUM  AMPLITUDE  CRITERION:  150  A/D  UNITS 
SACCADE  BIG  INCREASE/DECREASE  CRITERION:  150  A/D  UNITS 
SACCADE  WINDOW  CRITERION:  200  A/D  UNITS 
SAMPLING  RATE:  1  POINT  PER  10  MSECS 
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552. 

0.070 

0.040 

0.040 

0.070 

0.410 

0.080 

-242. 

-287. 

73 

61  >  87.68a> 

1.230 

0.050 

244. 

215. 

-59 

^47  87.680' 

1.350 

1097. 

0.150 

0.040 

0.050 

0.070 

48  88.060 

0.400 

797. 

0.130 

0.090 

0.050 

0.060 

53  >88.110 

0.380 

0.100 

-78. 

-316. 

60 

49  88.670 

1.590 

791. 

0.090 

0.060 

0.030 

0.050 

34  BUNKS  COXHCIOEHT  WITH  HORlZONtAL  SACCADCS 


SACCAOe  OUFAXIOH  H  X  9 


SACCADCS 

ASSOCIATED  W/BLIIIXS 

27 

82.6 

42.57 

t  a  3.468 

SACCAOCS 

WZTHOirr  BLXIIKS 

35 

117.1 

33.34 

31  Blink  Saccada  cooecurranca,  i.a.,  30  aaac  dala/  or  laas 
4  Blink  Saccada  non-occurranca ,  i.a.,  40  aaac  dalay  or  lon^ar 
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saccades,  or  may  be  produced  by  eitifacts,  such  as 
skin  potential  shifts.  Column  D  refers  to  the 
amplitude  of  the  saccade,  and  column  E,  "peak 
velocity,"  refers  to  the  largest  change  (in  A/D 
units)  occurring  during  a  saccade.  In  the  present 
instance,  it  is  the  largest  shift  occurring  between 
10  msec  samples  during  the  saccade  (since  sampling 
period  is  10  msec).  The  criterion  used  for  defining 
a  saccade  was  considerably  more  liberal  than  our 
usual  definition  of  a  saccade.  Thus,  both  relatively 
rapid  pursuit  and  compensatory  eye  movements  were 
accepted  as  saccades  in  this  analysis. 

1.2  Interpreting  Data  Output 

In  Table  3,  we  see  that  a  blink  occurred  560  msec  into 
this  digitization  run,  and  that  30  msec  later,  a 
horizontal  saccade  was  initiated  Since  the  closing 
duration  of  the  blink  was  120  msec  (column  7),  and  the 
saccade  duration  130  msec,  the  saccade  terminated  40  msec 
after  the  eyelid  started  to  reopen.  In  the  table,  we 
have  manually  identified  blinks  that  occurred 
concurrently  with  a  saccade.  Concurrence  is  defined  as  a 
time  lag  between  blink  and  saccade  initiation  of  80  msec 
or  less.  In  fact,  the  great  majority  of  bl ink-saccade 
latencies  fell  in  the  +  30  msec  range. 

The  column  on  the  left,  between  that  containing  the 
numbered  blinks  and  saccades,  and  the  one  identifying  the 
time  of  occurrence  of  the  numbered  events,  was  manually 
added.  This  column  deals  with  oblique  saccades,  which 
appear  in  the  record  as  simultaneous  horizontal  and 
vertical  saccades.  This  information  was  abstracted  by 
combining  the  horizontal  and  vertical  saccade  analysis. 

If  a  saccade  in  either  plane  was  initiated  within  60  msec 
of  that  in  the  other  plane,  the  two  saccades  were 
considered  to  be  coincident.  As  is  apparent  in  Table  3, 
not  only  does  the  first  blink  occur  concurrently  with  a 
saccade  in  the  horizontal  plane,  there  is  a  concurrent 
saccade  in  the  vertical  plane  (i.e.,  eye  position  in  the 
vertical  plane  following  the  blink  is  markedly  different 
from  eye  position  preceding  the  blink). 

1.3  Blink  and  Saccade  Data  Summa r i e s .  Table  4  summarizes 
information  about  the  blinks  abstracted  in  Table  3  and  is 
self-explanatory.  Table  5  requires  some  clarification: 
saccade  data  are  broken  down  into  25  columns  and  three 
levels.  Referring  to  the  third  section  on  saccade  ampli¬ 
tude,  the  first  row  breaks  amplitude  into  20  A/D  unit 
steps.  Thus,  row  1/column  1  includes  all  amplitudes  from 
0-19  msec,  row  1/column  2,  20-39  msec,  etc.  There  are  no 
entries  for  the  first  seven  columns,  because  our  criter¬ 
ion  for  defining  saccade  amplitude  (see  upper  right-hand 
corner  of  Table  3)  did  not  allow  for  the  identification 
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BLINK  SUW4ARY 


AMPLITUDE 

HALF  DURATION 

WINDOW  duration 

DESCENT 

(50%) 

CLOSING  DURATION 

A/D  UNITS 

FREQ 

MSECS 

FREQ 

MSECS 

FREQ 

MSECS 

FREQ 

MSECS 

FREQ 

200-  219 

1 

1-  10 

0 

1-  10 

0 

1-  10 

0 

1-  10 

0 

220-  239 

0 

11-  20 

0 

11-  20 

0 

11-  20 

1 

11-  20 

0 

240-  259 

0 

21-  30 

0 

21-  30 

0 

21-  30 

11 

21-  30 

0 

260-  279 

1 

31-  40 

0 

31-  40 

11 

31-  40 

20 

31-  40 

0 

280-  299 

0 

41-  50 

1 

41-  50 

14 

41-  50 

11 

41-  50 

2 

300-  319 

1 

51-  60 

1 

51-  60 

10 

51-  60 

4 

51-  60 

7 

320-  339 

0 

61-  70 

11 

61-  70 

6 

61-  70 

2 

61-  70 

18 

340-  359 

0 

71-  80 

6 

1 

OB 

o 

5 

71-  80 

0 

71-  80 

9 

360-  379 

1 

81-  90 

8 

81-  90 

3 

81-  90 

0 

81-  90 

5 

380-  399 

3 

91-100 

4 

91-100 

0 

91-100 

0 

91-100 

1 

400-  419 

1 

101-110 

3 

101-110 

0 

101-110 

0 

101-110 

3 

420-  439 

1 

111-120 

2 

111-120 

0 

111-120 

0 

111-120 

4 

440-  459 

2 

121-130 

1 

121-130 

0 

121-130 

0 

121-130 

0 

460-  479 

0 

131-140 

5 

131-140 

0 

131-140 

0 

131-140 

0 

480-  499 

3 

141-150 

4 

141-150 

0 

141-150 

0 

141-150 

0 

500-  519 

0 

151-160 

0 

151-160 

0 

151-160 

0 

151-160 

0 

520-  539 

1 

161-170 

2 

161-170 

0 

161-170 

0 

161-170 

0 

540-  559 

3 

171-180 

0 

171-180 

0 

171-180 

0 

171-180 

0 

560-  579 

0 

181-190 

1 

181-190 

0 

181-190 

0 

181-190 

0 

580-  599 

2 

191-200 

0 

191-200 

0 

191-200 

0 

191-200 

0 

over  599 

29 

OVER  200 

0 

OVER  200 

0 

OVER  200 

0 

OVER  200 

0 

MEAN 

748.08 

102.24 

57.76 

42.45 

78.78 

VARIANCE  105836.59 

1142.77 

226.11 

118.88 

340.14 

ST.  DEV. 

325.33 

33.80 

15.04 

10.90 

18.44 

MEDIAN 

679.00 

90.00 

50.00 

40.00 

70.00 

NOTE:  MEDIAN  CALCULATED  FROM  RAW  SCORES,  NOT  FROM  FREQUENCY  DISTRIBUTION. 
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TABLE  5 


20  MS.  STEPS:  0  0  0 

100  MS.  STEPS:  110 
500  MS.  STEPS:  11  18  11 


10  MS.  STEPS: 
20  MS.  STEPS: 


100  MS.  STEPS:  31  28 


9  IS  9  12 


20  A/D  STEPS:  0  0  0  0 

50  A/D  STEPS:  0  0  0  14 

100  A/D  STEPS:  0  14  Ifi  15 


SACCADE  SUMMARY 


FIXATION  DURATION: 


9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 


SACCADE  DURATION: 

9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 


SACCADE  AMPLITUDE: 

9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 


SACCADE  VELOCITY: 

9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 


2  A/D  STEPS:  0 
5  A/D  STEPS:  0 
10  A/D  STEPS:  0 


3  14  11  16 


The  correlation  coefficient  for  peak  velocity  vs.  amplitude  is  0.655. 

The  slope  of  the  regression  line  is  6.01,  and  the  y-intercept  is  -83.75. 


peak  velocity 


amplitude 


variance:  460.277  38768.016 

standard  deviation:  21.454  196.898 

standard  error  of  estimate:  16.204  148.713 

The  correlation  coefficient  for  saccade  duration  vs.  saccade  amplitude  is  0.585. 
The  slope  of  the  regression  line  is  2.80,  and  the  y-intercept  is  77.76. 

duration  amplitude 


standard  deviation: 
standard  error  of  estimate: 


38468.016 


of  saccades  smaller  than  150  A/D  units.  One  finds,  for 
example,  that  there  are  2  +  6  +  6  =  14  saccades  smaller 
than  200  A/D  units.  The  next  row  breaks  saccade  ampli¬ 
tude  into  50  msec  units,  again  with  14  units  following 
between  150  and  200  msec.  The  third  row  breaks  the  data 
down  into  100  A/D  unit  steps,  and  we  see  that  14  saccades 
are  between  100  and  200  A/D  units  in  amplitude.  Since 
saccade  amplitude  and  duration,  as  well  as  amplitude  and 
peak  velocity,  are  (under  laboratory  circumstances) 
highly  correlated,  these  individual  descriptors  of 
saccade  components  are  not  easily  interpreted. 

Because  of  these  relationships,  we  have  also  calculated 
the  linear  regression  of  saccade  amplitude  on  velocity, 
as  well  as  saccade  amplitude  on  duration. 

2.0  Blinks  and  Saccades  in  Simulated  Flight 

2.1  Ingress  mission 

A  typical  ingress  mission  flight  plan  is  presented  in 
Figure  7.  There  were  variations  on  this  theme  but  the 
basic  structure  of  the  flights  was  the  same.  Waypoints 
are  indicated  by  circled  numbers. 

Two  sets  of  data  were  evaluated  for  the  ingress  mission. 
The  first  dealt  with  oculometric  measures  collected 
around  the  waypoints.  For  this  analysis,  data  from  the 
60  second  period  prior  to  arrival  at  a  waypoint,  and  the 
60  second  period  following  crossing  of  the  waypoint  were 
analyzed.  We  occasionally  found  that  the  time  between 
the  first  and  second  waypoint  was  less  than  120  seconds. 
Where  this  occurred,  data  around  the  first  and  third 
waypoints  were  analyzed. 

The  second  segment  analyzed  dealt  with  an  announced 
threat,  which  always  occurred  following  the  turn  at 
waypoint  two  but  prior  to  crossing  waypoint  three.  After 
a  short  delay,  the  threat  was  declared  terminated.  In 
both  cases,  the  60  second  period  immediately  following 
the  announcement  was  analyzed. 

2.2  Refuel  mission 

Up  to  six  segments  of  60-sec  chunks  of  data  were 
abstracted.  These  were  coded  as  listed  for  clarity  of 
presentation;  however,  the  numbers  in  parentheses  refer 
to  the  segments  in  Figure  9. 

BT  (1)  -  60  s  period  prior  to  the  announcement  that 
the  tanker  had  started  his  turn. 

TT  (2,3)  -  60  s  period  after  the  tanker  initiated  his 
turn . 

PH  (4,5)  -  60  s  period  before  the  pilot  announced 
sighting  the  tanker  (Tally-ho). 

HO  (6,7)  -  60  s  period  immediately  following  the 


42 


sighting  of  the  tanker. 

RZ  (7+,8)  -  60  s  period  preceding  rendezvous  of  the 
two  aircraft. 

Where  no  RZ  data  are  identified,  it  occurred  within  90 
seconds  or  less  following  HO,  or  no  rendezvous  was  made. 

3.0  Summary  of  Results 

3.1  Blink  and  Saccade  Frequency.  Absolute  number  of  blinks 
and  saccades  were  expressed  as  rate  measures,  events/min¬ 
ute,  for  all  data  analyzed  for  subjects  1  and  5,  and  are 
depicted  in  Tables  6  and  7.  Results  are  as  follows: 

1.  For  Subject  1,  no  consistent  differences  in  blink 
rate  were  found  between  HMD  and  noHMD  conditions  (HMD  > 
noHMD  in  only  4  of  12  segments).  Subject  5  blinked 
consistently  more  under  the  noHMD  condition  (16  out  of  16 
segments  of  data  analyzed). 

2.  Both  subjects  made  more  horizontal  and  vertical 
saccades  (estimated  at  greater  than  2°)  in  the  noHMD  than 
in  the  HMD  condition  (in  all  four  missions  evaluated). 

3.  Subject  5,  for  whom  data  for  Threat  avoidance  and  a 
post-Threat  period  (return  to  course)  are  available, 
demonstrates : 

a)  a  marked  increase  in  horizontal  saccades  under  the 
Threat  condition,  with  maintenance  of  this  effect  through 
the  Post-Threat  period.  This  effect  is  seen  for  both  the 
HMD  and  NoHMD  conditions. 

b)  For  both  Threat  and  Post-Threat  conditions,  the 
vertical  saccade  pattern  is  the  reverse  of  the  horizontal 
pattern:  the  HMD  condition  produced  more  gaze  shifts  than 
the  noHMD  condition. 

3.2  Bl ink/Saccade  Concurrence .  The  last  column  in  Tables  6 
and  7,  labeled  "V  SAC  W/BLINK" ,  needs  additional  qualif¬ 
ication.  This  designation  refers  to  the  concurrence  of  a 
blink  with  a  vertical  saccade.  In  the  waveform  suggesting 
this  occurrence,  the  eye  position  (voltage  level) 
following  a  blink  is  markedly  different  from  that  prece¬ 
ding  the  blink.  A  voltage  level  lower  following  the  blink 
than  preceding  it,  indicates  that  the  gaze  has  shifted 
upward  (and  vice  versa). 

These  data  were  abstracted  for  two  reasons.  Of  major 
importance  was  that  our  algorithm  for  blink  identifica¬ 
tion,  as  mentioned  earlier,  will  not  identify  a  blink  as 
such  if  eye  position  following  a  blink  returns  to  a 
voltage  level  less  than  half  the  total  blink  amplitude 
(cf.,  point  F,  Figure  12c).  Thus,  blinks  for  which  gaze 
direction  went  up  during  the  course  of  the  blink,  would 
not  be  compute r- identi f ied  as  blinks.  Blinks  associated 
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Table  6 


Subject  1,  B  Mission 


EVENTS/MIN  EVENTS/MIN 

HMD  NOHMD 


V  SAC  V  SAC 


BLINKS 

SAC  H 

SAC 

V  W/BLINK 

BLINKS 

SAC  H 

SAC  V 

W/BLINK 

IW 

27 

35 

23 

1.5 

24.5 

67.5 

54 

9.5 

3W 

18 

32.5 

39 

6 

17 

52.5 

63 

5.5 

BT 

33 

27 

12 

4 

12 

42 

33 

2 

TT 

41 

59 

15 

2 

24 

— 

66 

6 

PH 

17 

70 

65 

4 

17 

65 

66 

9 

HO 

35 

30 

63 

13 

36 

46 

39 

11 

RZ 

49 

13 

49 

5 

35 

70 

27 

10 

Subject  1,  D  Mission 


EVENTS/MIN 

EVENTS/MIN 

HMD 

NOHMD 

V  SAC 

V  SAC 

BLINKS 

SAC 

H  SAC  V 

W/BLINK 

BLINKS 

SAC  H 

SAC  V 

W/BLINK 

IW 

24.5 

33 

48 

9.5 

28.5 

59 

63 

13 

3W 

17.5 

45 

45.5 

4 

26.5 

55.5 

59.5 

19.5 

TT 

39 

39 

26 

5 

25 

105 

70 

18 

PH 

47 

12 

46 

12 

35 

66 

35 

16 

HO 

34 

32 

71 

13 

36 

44 

47 

14 
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Table  7 


Subject  5,  A  Mission 


EVENTS/MIN 

HMD 


V  SAC 


BLINKS 

SAC 

H 

SAC  V 

W/BLINK 

iw 

17.5 

17 

18 

8.5 

2W 

18.5 

17. 

5 

24 

6 

TH 

34 

40 

71 

25 

PTH 

25 

36 

57 

21 

TT 

25 

13 

56 

17 

PH 

13 

8 

58 

12 

HO 

11 

0 

19 

3 

Subject 

EVENTS/MIN 

HMD 

V  SAC 

BLINK 

SAC  H  SAC  V 

W/BLINK 

iw 

23.5 

21.5  41.5 

12 

3W 

24 . 5 

21.5  33.5 

7.5 

TH 

21 

4  3  74 

12 

TH 

13 

27  60 

9 

EVENTS/MIN 


NOHMD 

V  SAC 

BLINKS 

SAC 

H  SAC  V 

W/BLINK 

29.5 

44 

50 

25 

24.5 

31 

28 . 5 

30 . 5 

37 

51 

61 

35 

38 

47 

71 

25 

33 

47 

62 

17 

25 

36 

97 

18 

17 

29 

66 

23 

,  C  Mission 

EVENTS/MIN 

NOHMD 

V  SAC 


BLINK 

SAC  H 

SAC  V 

W/BLINK 

36 

46 . 5 

49 . 5 

28 . 5 

33 

43  .  5 

65.5 

30 . 5 

33 

78 

39 

24 

27 

45 

64 

22 

45 


with  these  two  patterns,  thus,  are  blinks  where  eye 
position  following  blink  termination  is  radically 
changed.  In  other  contexts,  we  had  observed  that  such 
blinks  are  also  more  likely  to  be  associated  with  head 
movements  than  was  true  of  more  "conventional"  blinks. 

To  the  extent  that  the  pilot  has  to  acquire  information 
from  the  instrument  panel  under  NoHMD  conditions,  which, 
in  the  HMD  condition,  can  be  acquired  from  the  informa¬ 
tion  projected  onto  the  visor,  a  tally  of  these  blinks 
under  NoHMD  and  HMD  conditions  should  provide  us  with 
some  information  about  the  pilot's  use  of  HMD  displayed 
information . 

One  might  suspect  that  the  habit  of  checking  t  ,e  instru¬ 
ment  panel,  inculcated  by  thousands  of  hours  of  flying, 
would  be  difficult  to  abort,  especially  for  pilots  who 
have  not  had  experience  with  using  head-up  displays.  On 
the  contrary,  these  results  demonstrate  the  high  level  of 
adaptability  of  our  pilots  with  respect  to  the  use  of 
novel,  and  presumably  easier  and  more  efficient,  means  of 
obtaining  information  about  the  aircraft.  Both  pilots, 
under  all  conditions,  show  a  marked  inhibition  of  such 
blink  patterns  when  the  HMD  was  available. 

3 . 3  Bl ink/Saccade  Concurrence ;  Manual  Analyses . 

Based  on  prior  observations  (and  a  hunch),  we  entertained 
the  hypothesis  that  the  concurrence  of  blinks  and 
saccadic  eye  movements  is  more  likely  under  high  than 
under  low  work  load  conditions.  This  should  be  especial¬ 
ly  so  when  visual  load  is  high,  since  during  both  a  blink 
and  a  saccadic  /e  movement,  there  are  periods  of  time 
when  little  visual  information  can  be  acquired.  In  the 
case  of  a  blink,  of  course,  no  information  can  be 
acquired  while  the  pupil  is  obscured  by  the  eye  lid;  and 
with  respect  to  saccades,  there  is  marked  suppression  of 
visual  input  upon  saccade  initiation  (and  for  a  period 
preceding  and  following).  This  suppression,  in  fact,  may 
persist  beyond  the  point  of  saccade  termination.  For 
these  reasons,  the  coincidence  of  these  ocular  events 
would  maximize  input  time  and  thus  increase  efficiency. 

The  literature  on  this  phenomenon  has  been  restricted  to 
the  study  of  horizontal  saccades.  We  know  of  no  data  on 
saccade  suppression  associated  with  vertical  or  oblique 
saccades.  We  suspect,  however,  that  to  the  extent  that 
saccade  suppression  is  associated  with  central  (nervous 
system),  rather  than  peripheral  (intra-ocular),  mechan¬ 
isms,  the  duration  of  suppression  may  be  affected  by 
central  factors.  Bartz  (1966)  has  alluded  to  such  a 
possibility  in  his  brief  discussion  of  "pre-saccade" 
suppression,  the  suppression  of  visual  intake  prior  to 
the  initiation  of  a  saccade.  Although  it  has  not  been 
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studied,  we  may  speculate  that  such  pre-saccade 
suppression  is  affected  by  such  central  variables  as 
"expectancy"  and  complexity  of  the  information  that  is  to 
be  acquired.  For  example,  pre-saccade  suppression  may  be 
affected  by  the  number  of  alternative  locations  to  which 
the  eye  may  be  required  to  move  following  the  presenta¬ 
tion  of  a  signal  at  one  of  several  alternate  locations. 
These  speculations  are  relevant  to  our  concern  with  the 
acquisition  of  information  when  it  is  available  on  the 
HMD,  as  compared  to  when  such  information  has  to  be 
acquired  via  relatively  large  amplitude  saccades,  and, 
perhaps,  head  movements. 

To  the  extent  that  information  acquisition  in  the  absence 
of  the  HMD  is  more  difficult,  this  condition  should  be 
accompanied  by  a  greater  incidence  of  the  co-occurrence 
of  blinks  and  saccades.  Carrying  this  argument  further, 
we  might  predict  that  blinks  may  also  be  more  f'^equently 
associated  with  oblique  saccades  in  the  NoHMD  .ondition. 
To  explore  these  possibilities,  we  have  initiated  an 
analysis  of  our  abstracted  data  (blinks  and  saccades)  for 
the  coincidence  of  these  events.  As  we  develop  our 
metric  for  this  purpose,  we  will  add  coincidence  of  head 
movement  to  these  measures. 

Since  there  were  marked  differences  between  the  HMD  and 
NoHMD  conditions,  in  the  total  number  of  blinks  and  the 
total  number  of  saccades,  we  expressed  the  frequency  of 
coincident  blink/saccades  in  each  condition  as  a  propor¬ 
tion  of  the  total  number  of  saccades  in  that  condition. 
This  was  done  separately  for  horizontal  and  vertical 
saccades.  Ideally,  such  an  analysis  would  have  separated 
blinks  associated  with  horizontal,  vertical,  and  oblique 
saccades.  Because  of  constraints  in  time  and  analytic 
procedure,  however,  this  was  not  done.  There  is, 
therefore,  overlap  between  categories,  i.e.,  the  same 
blinks  associated  with  oblique  saccades  also  appear  as 
occurring  with  horizontal  and/or  vertical  saccades. 

For  exploratory  purposes,  we  conducted  such  an  analysis 
for  Subject  5,  ingress  flight  C,  which  involved  a  Threat 
under  both  HMD  and  NoHMD  conditions.  These  results  are 
presented  in  Table  8.  The  column  headings  require  some 
explanation:  The  first  column,  labeled  HSAC/IH,  refers 

to  the  percentage  of  horizontal  saccades  accompanied  by  a 
blink.  For  the  first  and  second  waypoints  ( IW  and  2W )  the 
duration  of  the  periods  was  120  s,  60  s  before  and  60  s 
after  the  waypoint,  whereas  for  Threat  and  Post-Threat, 
it  was  only  the  60  s  period  following  the  announcements 
of  these  events. 

For  the  first  HMD  waypoint  (IW),  32%  of  horizontal 
saccades  had  an  associated  blink,  while  in  the  NoHMD 
condition,  that  figure  was  61%.  In  the  second  column  the 
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TABLE  8 


CO-INCIDENCE  OF  SACCADES  AND  BLINKS  UNDER 
HMD  AND  NoHMD  CONDITIONS 


HMD 


H  SAC 

V  SAC 

H  SAC 

V  SAC 

£  H 

£  V 

£  B 

£  B 

iw 

32 

29 

30 

51 

2W 

30 

21 

27 

29 

TH 

28 

16 

57 

57 

PT 

44 

15 

92 

69 

BT 

29 

22 

18 

50 

TT 

21 

14 

21 

57 

PH 

8 

10 

8 

33 

HO 

17 

5 

8 

15 

RZ 

33 

4 

15 

15 

COINC  SAC 

OBLIQUE  SACC 
W/BLINKS 

H+V  SAC 

H+V  SAC 

17 

38 

15 

50 

17 

43 

22 

47 

11 

2 

10 

1 

19 

2 

4 

—  — 

5 

4 

NoHMD 


H  SAC 

V  SAC 

H  SAC 

V  SAC 

COINC  SAC 

£  H 

£  V 

£  B 

£  B 

H+V  SAC 

iw 

61 

53 

79 

72 

25 

2W 

34 

47 

46 

92 

27 

TH 

22 

62 

51 

73 

19 

PT 

47 

30 

78 

70 

18 

BT 

46 

38 

67 

70 

27 

TT 

55 

36 

82 

74 

21 

PH 

58 

22 

73 

47 

15 

HO 

31 

16 

45 

59 

19 

RZ 

41 

15 

65 

57 

18 

OBLIQUE  SACC 
W/BLINKS 
H+V  SAC 

75 

48 

55 

55 

16 

15 

10 

7 

6 
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same  transformation  is  performed  for  blinks  associated 
with  vertical  saccades.  (We  should  point  out  that  we  are 
using  the  term  "saccade"  rather  loosely  here,  since  our 
criterion  for  saccade  identification  allowed  eye 
movements  considerably  slower  than  we  generally  require 
of  saccades,  e.g.,  pursuit  and  glissade  movements.)  In 
order  to  include  saccades  that  would  otherwise  be 
confounded  with  blinks,  as  described  above,  we  identified 
as  vertical  saccades,  a  shift  in  eye  position  following 
blink  termination  perceptibly  different  from  the  position 
preceding  blink  initiation.  We  estimate  that  if  the 
smallest  saccade  accepted  by  our  computer  algorithm  was 
2“  of  arc,  then  we  accepted  as  meeting  our  criterion  any 
vertical  position  shift  exceeding  4°  during  a  blink. 

The  second  column  performs  the  same  analysis  for  blinks 
co-incident  with  vertical  saccades  (as  defined  above). 
Again,  for  IW,  the  percentage  of  co-incident  blinks  and 
saccades  is  considerably  greater  in  the  NoHMD  than  in  the 
HMD  condition. 

The  next  two  columns  (3  &  4)  present  the  percentage  of 
coincident  bl inks/saccades ,  using  total  number  of  blinks 
in  the  denominator.  Whereas  the  earlier  measure 
represented  that  proportion  of  (all)  saccades  that  were 
accompanied  by  blinks,  this  value  is  the  proportion  of 
(all)  blinks  that  were  accompanied  by  saccades.  Column  5 
evaluates  oblique  saccades  (i.e.,  the  coincidence  of 
horizontal  and  vertical  saccades)  by  dividing  such 
coincident  saccades  by  the  total  number  of  horizontal  and 
vertical  saccades.  The  last  column  again  concerns 
blinks:  the  ratio  of  blinks  that  co-occur  with  coincident 
horizontal  and  vertical  saccades,  divided  by  the  sum  of 
all  horizontal  and  vertical  saccades. 

Referring  to  Table  8,  the  analysis  of  the  two-minute 
periods  around  waypoints  1  and  2  indicates  that  in  all  12 
cases  the  percentages  are  higher  for  the  NoHMD  (lower 
section  of  table)  than  for  the  HMD  condition  (upper  sec¬ 
tion).  The  five  refuel  segments  show  similar  unanimity 
in  favor  of  the  NoHMD  condition.  For  the  Threat  and  Post 
Threat  conditions,  we  find  the  effect  only  slightly  less 
uniform;  in  9  of  the  12  comparisons,  the  NoHMD  percent¬ 
ages  are  larger  than  the  HMD  percentages. 

4.0  Conclusions 

What  conclusion  do  we  draw  from  this  analysis?  If,  as  we 
suspect,  the  coincidence  of  blinks  and  saccades  is 
related  to  task  complexity,  then,  for  this  subject,  we 
can  infer  that  flying  under  the  NoHMD  condition  is  a  more 
complex  or  higher  work  load  task.  The  higher  blink  rate 
under  the  NoHMD  condition  cannot  account  for  this  effect, 
since  it  is  effectively  counterbalanced  by  an  even 
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greater  increase  in  saccades  in  both  the  horizontal  and 
vertical  planes  (see  Table  7). 

We  also  have  suggestive  evidence  that  such  coincidence  of 
blinks  with  saccades  is  more  prevalent  with  oblique  than 
single  plane  saccades  (compare  the  last  column  ot  Table  8 
with  the  first  two  columns.)  We  believe  that  the  actual 
differences  in  these  percentages  would  have  been  larger 
had  data  for  oblique  saccades  not  also  been  included  in 
the  analyses  involving  the  horizontal  and  vertical 
saccades . 

We  had  expected  that  the  Threat  and  Post-Threat  condi¬ 
tions  might  impose  higher  work  load  on  the  pilot  than  the 
navigational  checks  and  course  changes  associated  with 
crossing  of  waypoints.  Our  data  do  not  indicate  this  to 
be  the  case.  The  coincidence  percentages  are  not 
uniformly  higher  under  Threat  than  waypoint  conditions. 


Tables  9  and  10  take  the  above  analysis  one  step  further 
by  deriving  a  single  summary  meas'^re  directly  comparing 
the  HMD  and  NoHMD  percentages  as  they  had  been  derived 
for  Table  8.  The  summary  measure  contains  the  percentage 
for  the  NoHMD  condition  in  the  numerator  and  the  sum  of 
the  two  percentages  in  the  denominator.  Thus,  a 
resultant  value  in  excess  of  50  indicates  a  higher 
percentage  for  the  NoHMD  condition.  This  analysis  was 
conducted  for  two  ingress  and  two  refuel  missions  for 
subject  1  (who  had  no  threats  presented  on  any  of  his 
ingress  missions)  and  two  ingress  missions  with  threats, 
as  well  as  two  refuel  missions  for  subject  5. 


For  all  four  ingress  missions,  the 
demonstrate  consistently  higher  ra 
as  compared  to  the  HMD  conditions, 
obtained  for  each  mission,  15  and 
Subject  1,  and  15  and  18  exceed  50 
interpret  these  results  as  indicat 
higher  under  the  NoHMD  condition. 


waypoint  segments 
tios  under  the  NoHMD, 
Of  the  18  measures 
18  exceed  50%  for 
%  for  Subject  5.  We 
ing  that  work  load  is 
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TABLE  9 


COMPARISON  OF  NoHMD  WITH  HMD  CONDITIONS  FOR  SUBJECT  1 
(NOHMD/NOHMD  +  HMD)  X  100 


B  Mission 


£ 

BLINKS 

£ 

SACCADES 

HOR  VER 

COINC  BL 
£  SACC 

HOR  VER 

COINC  BL 
£  BLINKS 
HOR  VER 

COINC  H+VS 
£  SACC 

V  SAC 
W/BLINKS 

IW  48 

66 

69 

63 

90 

78 

96 

64 

86 

3W  49 

62 

62 

61 

56 

72 

69 

63 

48 

BT 

27 

61 

73 

21 

40 

45 

77 

68 

33 

TT 

37 

— 

81 

— 

75 

— 

58 

— 

75 

PH 

50 

48 

50 

46 

68 

44 

69 

50 

69 

HO 

51 

61 

38 

57 

69 

65 

58 

59 

46 

R2 

42 

84 

36 

46 

79 

87 

73 

68 

67 

D  Mission 


£ 

£ 

COINC  BL 

COINC  BL 

COINC  H+Vs 

BLINKS 

SACCADES 

£ 

SACC 

£  BLINKS 

£  SACC 

V  SAC 

HOR 

VER 

HOR 

VER 

HOR 

VER 

W/BLINKS 

IW  54 

64 

57 

72 

60 

72 

60 

62 

58 

3W  6  0 

55 

57 

65 

75 

65 

75 

64 

83 

TT  39 

73 

73 

46 

48 

46 

48 

61 

78 

PH  43 

85 

43 

58 

43 

58 

43 

64 

57 

HO  51 

58 

40 

79 

54 

79 

54 

57 

52 

51 


TABLE  10 


COMPARISON  OF  NoHMD  WITH  HMD  CONDITIONS  FOR  SUBJECT  5 
(NOHMD/NOHMD  +  HMD)  X  100 

A  Mission 


E 

E 

COINC 

BL 

COINC 

BL 

COINC  H+Vs 

BLINKS 

SACCADES 

E  SACC 

E  BLINKS 

E  SACC 

V  SAC 

HOR 

VER 

HOR 

VER 

HOR 

VER 

W/BLINKS 

IW  63 

72 

74 

40 

39 

56 

66 

39 

75 

2W  57 

62 

54 

67 

77 

74 

72 

74 

84 

TH  57 

78 

53 

47 

74 

70 

36 

66 

58 

PH  60 

57 

55 

41 

43 

59 

45 

43 

54 

TT  57 

78 

53 

47 

74 

70 

36 

66 

50 

PH  66 

82 

63 

52 

74 

82 

56 

70 

60 

HO  61 

100 

78 

— 

— 

-- 

63 

— 

89 

C  Mission 

E 

E 

COINC 

BL 

COINC 

BL 

COINC  H+Vs 

BLINKS 

SACCADES 

E  SACC 

E  BLINKS 

E  SACC 

V  SAC 

HOR 

VER 

HOR 

VER 

HOR 

VER 

W/BLINKS 

IW  61 

68 

70 

66 

65 

72 

59 

60 

70 

2W  57 

67 

66 

53 

69 

63 

76 

64 

80 

TH  61 

64 

35 

44 

79 

47 

56 

53 

66 

PT  68 

63 

52 

52 

67 

46 

50 

45 

71 

BT  55 

74 

51 

61 

63 

79 

58 

77 

63 

TT  71 

78 

55 

72 

72 

80 

56 

68 

62 

PH  71 

76 

60 

91 

69 

90 

59 

44 

62 

HO  6  3 

84 

67 

65 

76 

85 

80 

83 

87 

RZ  64 

86 

63 

55 

79 

81 

79 

78 

90 
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D.  HEAD  AND  EYE  MOVEMENTS;  LITERATURE  REVIEW 


Timing  of  Head  and  Eye  Movements 

The  literature  on  the  relationship  between  head  and  eye 
movements  under  relatively  "normal"  conditions  is  scarce 
indeed.  If  by  "normal"  conditions  we  mean  situations 
involving  the  dividing  of  attentional  and  other  resources 
between  a  number  of  tasks,  only  one  published  study  meeting 
such  criteria  can  be  found  (Mourant  &  Crimson,  1977).  If  we 
include  conditions  where  performance  of  a  single  task  is 
required,  we  find  a  few  more  (Bartz,  1966;  Barnes,  1977; 
Bizzi,  1974;  Calhoun,  Arbak  &  Janson,  1986;  Gresty,  1974; 
Robinson,  1979;  and  Robinson,  Koth  &  Ringenbach,  1976). 

Mourant  and  Crimson  (1977)  collected  eye  and  head  movements 
of  subjects  driving  under  normal  highway  traffic  conditions. 
They  focused  on  the  use  of  rear  view  mirror  gazes.  Two 
patterns  of  mirror  gaze  were  described:  a  "classical"  and  a 
"predictive"  pattern.  The  classical  pattern  involved  eye 
movements  initiated  prior  to  head  movements,  while  in  the 
predictive  pattern,  head  movements  preceded  eye  movement 
initiation.  They  obtained  approximately  twice  as  many 
predictive  than  classical  patterns  in  their  nine  subjects. 

The  average  latency  for  the  classical  pattern  was  approxi¬ 
mately  45  msec,  a  latency  that  is  comparable  to  that  reported 
by  Bizzi  (1974)  for  the  monkey.  For  the  predictive  pattern, 
they  report  an  average  latency  of  90  msec  between  head  and 
eye  movement  initiation.  On  the  basis  of  data  presented  in 
the  paper,  it  appears  to  us  that  the  90  ms  latency  reported 
is  not  representative;  it  overweights  data  collected  in  one 
of  the  four  conditions  of  the  experiment.  That  is,  average 
latency  for  each  of  the  four  conditions  was  66,  161,  64  and 
68  ms.  Excluding  the  "aberrant"  condition,  the  average  is  66 
ms,  which,  perhaps,  is  a  better  basis  for  estimating 
predictive  latency.  Nevertheless,  Mourant  &  Crimson  quote  an 
unpublished  paper  by  Robinson  and  Subelman  (1975)  involving 
dual  task  performance,  where  subjects  had  to  monitor  a 
display  at  right  angles  to  a  central  control  task.  These 
authors  report  that  63%  of  the  head-eye  movement  patterns 
were  predictive  in  nature  with  a  latency  of  117  ms.  Clearly, 
the  conditions  favoring  the  predictive  pattern,  and  the 
saccade/head  movement  time  relations  involved,  are  still 
unresolved . 

Studies  involving  the  measurement  of  eye  and  head  movements 
where  task  requirements  are  simple,  most  frequently  report 
the  classical  pattern.  Extrapolating  from  the  figure 
presented  in  Bartz  (1966),  it  appears  that  the  head  movement 
follows  eye  movement  initiation  by  approximately  100  msec. 

Gresty  (1974)  does  not  report  the  time  difference  between 
saccade  and  head  movement  initiation.  Extrapolating  from  his 
average  values,  head  movements  are  initiated  approximately  68 
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msec  after  saccade  initiation.  Robinson  (1979)  reports  that 
latencies  range  between  0  and  50  msec,  while  Robinson,  Koth, 

&  Ringenbach  (1976)  report  latencies  of  approximately  50 
msec . 

Only  one  study,  Calhoun,  Arbak  &  Janson  (1986),  reports 
latencies  considerably  longer  than  the  above.  Utilizing  a 
task  where  head  and  eye  movements  were  in  the  VERTICAL  plane 
(all  of  the  above  mentioned  studies  dealt  only  with  horizon¬ 
tal  plane  movements),  these  authors  report  latencies  of  440 
ms  between  eye  and  head  movement  initiation.  They  suggested 
that  "system  delay"  in  calculating  eye/head  position  shifts 
can  be  on  the  order  of  200  ms.  However,  subtracting  200  from 
440  still  leaves  their  latencies  outside  the  range  reported 
in  the  other  studies  reviewed  here.  These  authors  also  report 
on  the  occurrence  of  the  predictive  pattern  in  17%  of  trials. 
However,  70%  of  the  data  were  contributed  by  one  subject  (N  = 
8).  Gresty  (1974)  reported  that  one  of  his  subjects  demon¬ 
strated  the  predictive  pattern,  with  saccadic  eye  movements 
following  head  movement  initiation,  by  about  40  ms.  This 
subject  wore  contact  lenses,  and  attributed  this  pattern  to 
"...the  fact  that  if  she  made  fast  eye  movements,  her  contact 
lenses  might  fall  out."  (p.  399). 

From  these  studies,  one  can  conclude  that,  in  general,  gaze 
shifts  involve  the  initiation  of  saccades  approximately  50  ms 
before  the  head  starts  to  move.  Robinson,  Koth  and 
Ringenbach  (1976)  correlated  saccade  and  head  movement 
latencies,  with  respect  to  stimulus  presentation.  The 
average  correlation  was  0.92,  suggesting  the  high  likelihood 
that  both  are  controlled  by  the  same  central  nervous  system 
center,  a  conclusion  also  arrived  at  by  Vossius  (1972). 

That  the  pattern  of  eye  and  head  movements  cannot  be 
completely  accounted  for  by  simple  quantitative 
bioengineering  variables  is  well  attested  to  in  the 
literature  reviewed  above.  For  example,  Robinson,  Koth  & 
Ringenbach  (1976)  report  that  the  pattern  of  eye  and  head 
movements  is  affected  by  eye  movement  calibration  procedures. 
"It  might  be  noted  here,  that  initial  eye  calibration 
procedures  used  a  bite  bar  and  the  procedure  was  found  to 
inhibit  head  movements  on  later  trials  after  it  had  been 
removed."  Robinson  (1979)  points  out  that  the  likelihood  of 
occurrence  of  head  movements  in  the  acquisition  of  peripheral 
information  is  affected  not  only  by  the  angular  distance 
between  current  eye  position  and  the  target  display  but  also 
by  the  time  necessary  to  acquire  information  from  the 
peripheral  display,  and  by  the  anticipation  of  gaze  shifts  to 
other  displays.  In  a  similar  vein,  we  have  observed  that  the 
pattern  of  eye-head  coordinations  may  be  markedly  different 
ciepending  on  whether  information  can  be  acquired  rapidly,  or 
is  time  consuming,  or  whether  the  movement  returns  gaze  to  a 
central  fixation  position  following  peripheral  information 
acquisition.  Thus,  the  nature  of  the  information  to  be 
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processed,  as  well  as  immediately  preceding  task  require¬ 
ments,  may  be  as  important  in  determining  the  patterning  of 
eye-head  movements  as  the  biomechanical  control  mechanisms 
affecting  gaze  shifts. 

Our  studies  provide  some  additional  information  concerning 
eye-head  movement  dynamics.  In  these  studies,  subjects  had 
to  acquire  information  from  peripheral  locations  to  the  left 
(L)  or  right  (R)  of  the  central  fixation  point.  Specifically, 
the  locations  were  45“  L,  15°  L,  0°,  15“  R,  and  45“  R.  Two 
levels  of  task  requirements  were  utilized.  One  involved  the 
detection  of  a  peripheral  signal,  a  letter,  following  a 
centrally  presented  letter.  In  the  "Detect"  condition,  the 
subject  was  required  to  indicate  whether  or  not  a  peripheral 
signal  had  been  presented  following  termination  of  the 
central  signal.  In  the  "Identification"  condition,  he  was  to 
determine  whether  the  peripherally  presented  letter  was  the 
same  as  or  different  than  the  centrally  presented  one. 

Under  the  Detect  condition,  we  obtained  very  few  eye  move¬ 
ments  (prior  to  responding),  and  no  head  movements.  Under 
the  Identification  condition,  there  were  eye  movements  to 
foveate  all  peripheral  targets,  but  few  head  movements. 

Based  on  the  work  by  Ganders,  we  had  expected  that  most  of 
the  trials  involving  target  eccentricities  of  45“  would  be 
accompanied  by  head  movements.  On  trials  where  head  movements 
did  occur,  the  average  latency  of  head  movements  (following 
onset  of  the  peripheral  target)  was  50  ms  after  saccade 
initiation.  Head  movement  returning  gaze  to  the  central 
location,  in  anticipation  of  the  next  centrally  presented 
signal,  was  much  more  variable  with  respect  to  eye  movements. 
Head  movement  velocities  appeared  to  be  considerably  slower, 
and  head  movement  initiation  frequently  preceded  eye  movement 
initiation  by  100  ms  or  more.  We  would  be  loath  to  call  the 
latter  head  movements  "predictive,"  in  the  sense  that  Mourant 
and  Crimson  (1977)  used  that  term.  Regardless  of  what  one 
wishes  to  call  these  head  movements,  the  fact  that  there  is 
more  than  one  pattern  of  coordination  of  eye  and  head 
movements,  rules  out  simplistic  explanations  of  such 
activity,  and  stresses  the  possibility  that  their  temporal 
patterning  can  be  used  to  make  inferences  about  aspects  of 
information  processing. 

A  second  variable  of  concern  deals  with  the  duration  of  head 
movements.  This  parameter  has  been  investigated  in  two 
studies.  Zangeme i ste r ,  Jones  &  Stark  (1981)  evaluated  head 
movements  under  conditions  where  subjects  were  instructed  to 
move  their  heads  (not  gaze)  as  rapidly  as  possible  between 
targets  that  were  from  2“  to  140°  apart  (no  eye  movement  data 
were  collected  in  this  study).  Gresty  (1974)  reported  on 
head  movement  duration  to  stimuli  presented  at  eccentricities 
up  to  81.7°  left  and  right  of  center.  Both  studies  report 
head  displacement  durations  of  about  450  ms,  with  duration 
invariant  with  respect  to  amplitude.  What  one  can  see  in  the 
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Zangemeister ,  et  al . ,  paper  is  that  head  movement  duration  is 
more  variable  for  smaller  amplitude  movements. 
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E. 


HEAD  MOVEMENT  ANALYSIS 


MANUAL 


1.0  General  information 

Head  movement  data  collection  utilized  the  Polhemus  head 
tracking  system.  As  utilized  in  the  present  application, 
HORIZONTAL  and  VERTICAL  plane  head  movements  were  digitized 
at  a  sampling  rate  of  20/sec  (50  ms  sampling  period).  They 
were  then  D/A  converted  for  storage  on  magnetic  tape.  Two 
channels  of  head  movement  data  were  thus  used  in  the  current 
analysis . 

Data  for  all  subjects  were  strip  charted.  Both  horizontal 
and  vertical  eye  movements  and  horizontal  and  vertical  head 
movements  were  charted  concurrently.  Relevant  voice 
information,  where  it  was  intelligible,  was  identified  on  the 
strip  charts. 

Segments  of  data  for  each  subject  were  digitized  at  a  rate  of 
100  or  200  samples/sec  and  stored  on  disks.  For  our  initial 
head  movement  analysis,  segments  of  data  with  readily 
discernible  head  movements  were  digitized  for  subsequent 
analysis . 

Data  reduction  for  all  data  utilized  our  MARLAB  program.  The 
analysis  of  all  head  movement  data  utilized  manual  point 
setting  routines  to  identify  initiation  and  termination  of 
head  movements,  as  well  as  the  amplitude  of  such  movements. 
Where  head  and  eye  movement  and  blink  information  was  to  be 
abstracted,  we  either  used  a  manual  point  setting  routine  for 
all  four  channels  of  data,  or  the  oculometric  analysis  was 
conducted  semi-automatically . 

The  semi-automatic  analysis  involves  defining  parameters  that 
are  to  be  used  for  defining  vertical  EOG  activity  as  eye 
blinks,  and  a  second  set  of  parameters  to  define  horizontal 
EOG  activity  as  saccades.  These  algorithms  are  then  applied 
to  the  data  displayed  on  a  monitor  (1000  data  points  at  a 
time),  and  the  operator  can  perform  edit  functions  on  such 
data . 

The  integration  of  eye  and  head  movement  data  was  done 
manually.  The  data  reduction  process  (at  this  time)  is, 
thus,  a  slow  and  labor-intensive  effort. 

2.0  Selection  and  Categorization  of  Data 

The  analyses  described  in  this  section  were  done  manually; 
data  for  all  flights  were  strip  charted  and  head  movements 
were  identified  by  examining  the  strip  chart.  For  each 
flight,  we  identified  head  movements  in  the  horizontal  and 
vertical  plane,  as  well  as  the  conjoint  occurrence  of  both 
horizontal  and  vertical  movements.  Our  criterion  for 


57 


including  a  head  movement  in  this  analysis  was  that  it  exceed 
10“  of  arc  (estimated)  in  either  the  horizontal  or  vertical 
plane . 

The  number  of  head  movements  (vertical,  horizontal,  and/or 
conjoint)  for  both  the  INGRESS  and  REFUEL  missions  were 
abstracted  and  summarized  based  on  the  following  patterns: 


1 . 

UP  only  (U) 

5. 

RIGHT 

and 

UP  (R+U) 

2. 

DOWN  only  (D) 

6. 

RIGHT 

and 

DOWN  (R+D 

3. 

RIGHT  only  (R) 

7. 

LEFT 

and 

UP  (L+U) 

4. 

LEFT  only  (L) 

8. 

LEFT 

and 

DOWN  (L+D) 

3,0  Presentation  and  Analysis  of  Data 

The  data  are  presented  in  two  formats  since  the  absolute 
number  of  head  movements  differed  markedly  both  across 
subjects  and  across  conditions  (HMD  or  noHMD;  INGRESS  or 
REFUEL  missions).  First,  the  total  number  of  all  head 
movements  for  each  of  the  five  subjects  under  each  of  the 
four  conditions  are  presented  in  Table  11.  Since  the  pilots 
did  not  necessarily  fly  an  equal  number  of  missions,  the 
totals  are  "corrected"  for  the  number  of  missions  flown. 

We  were  interested  in  determining  whether  there  was  anything 
systematic  about  the  distribution  of  head  movement  patterns 
when  frequency  of  head  movements  differed  as  a  function  of 
condition.  If  the  number  of  head  movements  under  the  noHMD 
condition  was  twice  as  large  as  under  the  HMD  condition,  for 
example,  we  were  interested  in  determining  whether  the 
increase  was  spread  equally  across  the  eight  head  movement 
patterns,  or  were  certain  patterns  favored? 

Two  analyses  were  conducted.  The  results  of  the  first, 
depicted  in  the  lower  portion  of  the  bar  graphs  (see  Figures 
13  -  22),  expressed  activity  in  each  of  the  eight  patterns  as 
a  percentage  of  the  sum  of  all  head  movements  within  either 
HMD  or  noHMD  conditions.  The  amplitudes  for  any  one  condit¬ 
ion  should  sum  to  100%. 

The  results  of  the  second  analysis,  depicted  in  the  upper 
right  hand  corner  of  the  bar  graphs  (see  Figures  13  -  22), 
depicts  summary  data,  again  using  the  sum  of  all  head 
movements  as  the  denominator.  The  numerator  for  TOTAL  RIGHT 
included  R,  R+U,  R+D;  for  TOTAL  LEFT,  the  numerator  included 
L,  L+U,  L+D;  for  TOTAL  UP,  the  numerator  included  U,  L+U, 

R+U;  and  for  TOTAL  DOWN,  the  numerator  included  D,  L+D,  R+D. 
Since  the  same  items  are  included  in  a  number  of  the  TOTAL 
values,  these  amplitudes  do  not  sum  to  100%. 
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SUBJECT  2;  HMD  vs.  NO  HMD;  REFUEL 


HEAD  MOVEMENT  ANALYSIS 

SUBJECT  B:  HMD  vs.  HO  HMD;  INGRESS 
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RIGHT  LEFT  RIGHT  RIGHT  LEFT  LEFT 


HEAD  MOVEMENT  ANALYSIS 

SUBJECT  3;  HMD  NO  HMD;  INGRESS 
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DOWN  RIGHT  LEFT  RIGHT  RIGHT  LEFT  LEFT 


TOTAL  TOTAL  TOTAL 

RIGHT  LEFT  UP 


HEAD  MOVEMENT  ANALYSIS 

SUBJECT  5;  HMD  vs.  NO  HMD;  INGRESS 
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4,0  Results 


4.1  Results  Based  Upon  Total  Head  Movements 

Examination  of  Table  11  makes  it  readily  apparent  that 
there  are  marked  individual  differences  in  the  frequency 
of  head  movements  across  subjects  (ranging  from  11  to 
220),  and  that  the  effect  of  the  four  conditions  is  not 
uniform  across  the  five  subjects.  The  first  two 
subjects  demonstrate  a  consistent  pattern  in  that  they 
make  more  head  movements  under  the  noHMD  than  the  HMD 
condition,  and  make  more  head  movements  in  the  REFUEL  as 
compared  to  the  INGRESS  flights. 

Subject  4  shows  a  similar  pattern,  with  more  head 
movements  under  the  noHMD  than  the  HMD  condition. 
However,  this  subject  differs  from  subjects  1  and  2  in 
that  he  makes  more  head  movements  during  INGRESS,  as 
compared  to  REFUEL  missions. 

Subjects  3,  4,  and  5  show  a  common  pattern,  in  that  they 
engage  in  more  head  movements  during  the  INGRESS  than 
the  REFUEL  missions. 

The  rather  obvious  conclusions  which  can  be  drawn  from 
this  rather  crude  analysis  are  that: 

1)  there  are  marked  individual  differences  in  the  use 
of  head  movements  associated  with  both  INGRESS  and 
REFUEL  flights. 

2)  some  pilots  (three  out  of  five)  make  markedly  more 
head  movements  under  the  noHMD,  as  compared  to  the 
HMD  condition. 

We  had  expected  that  the  HMD  condition  would  lead  to 
more  head  movements  since,  under  this  condition,  the 
pilots  would  engage  in  more  extensive  search  of  their 
environment,  having  the  normally  stationary  instrument 
information  available  on  their  visors  at  all  times;  this 
did  not  occur.  We  suspect  that  the  reduction  in  head 
movements  for  the  HMD  condition  reflects  the  fact  that 
these  pilots  did  make  use  of  the  information  displayed 
on  the  HMD,  but  did  not  use  the  "released  time"  to 
engage  in  other  visual  search  activity.  Thus,  pilots 
rapidly  learn  to  make  use  of  the  information  provided  on 
the  HMD.  It  is  our  suspicion  that  considerably  more 
experience  in  flying  with  the  HMD  must  occur  before  a 
pilot  starts  using  the  additional  time  available  to 
engage  in  new  patterns  of  visual  search  activity. 
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4.2  Results  Based  Upon  Relative  Head  Movement  Frequency 

4.2.1  Head  Movement  Patterns  During  INGRESS  Flights 

Regardless  of  availability  or  non-availability  of  the 
HMD,  the  most  frequent  head  movements  were  to  the 
right  or  right  coupled  with  a  downward  gaze.  Four  of 
the  five  subjects  made  a  greater  percentage  of  RIGHT 
ONLY  head  movements  under  the  noHMD  condition. 
Subjects  1  and  2,  who  showed  the  greatest  increase  in 
head  movements  from  the  HMD  to  the  noHMD  condition, 
demonstrate  a  markedly  higher  percentage  of  right 
head  movements  under  the  noHMD  condition. 

Analyses  demonstrate  that  subjects  3,  4,  and  5  make 
the  largest  proportion  of  their  head  movements  to  the 
RIGHT  AND  DOWN  (presumably  associated  with  looking  at 
their  maps),  but  the  differences  between  the  HMD  and 
noHMD  condition  in  this  gaze  shift  is  relatively 
small,  suggesting  that  availability  of  the  HMD  had  no 
effect  on  the  frequency  with  which  the  pilot  referred 
to  his  map  during  the  flight.  Two  pilots,  3  and  4, 
make  more  than  half  of  their  head  movements  utilizing 
this  pattern.  The  other  item  of  note  is  the  absence 
of  pure  UP,  pure  DOWN,  and  head  movements  to  the  LEFT 
AND  DOWN,  during  INGRESS  missions.  Again,  there  are 
no  differences  between  the  HMD  and  noHMD  conditions. 

One  can  conclude,  then,  that  for  those  subjects  who 
make  more  head  movements  in  the  noHMD  condition,  the 
majority  of  these  head  movements  involve  gaze  shifts 
in  the  horizontal  plane  to  the  right.  Whether  or  not 
the  HMD  is  available  does  not  affect  gaze  shifts  to 
the  map.  Further,  there  appear  to  be  marked  differ¬ 
ences  between  pilots,  with  respect  to  the  frequency 
with  which  they  refer  to  their  maps. 

4.2.2  Head  movement  patterns  during  REFUEL  flights 

The  head  movement  patterns  for  these  missions  were 
radically  different  from  those  in  the  INGRESS 
missions.  In  general,  as  one  might  expect,  there 
were  very  few  gazes  to  the  RIGHT  AND  DOWN,  the  gaze 
shift  associated  with  looking  at  the  map. 
Additionally,  comparison  of  INGRESS  and  REFUEL 
missions  demonstrated  that  right  head  movements  were 
not  as  dominant  a  pattern  during  REFUEL  missions  as 
was  true  for  the  INGRESS  missions  -  left  movements 
were  as  prominent  as  right  movements. 

There  were  no  consistent  patterns  of  head  movements 
that  discriminated  be  tween  HMD  and  noHMD  conditions 
across  subjects. 
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TABLE  11 


HEAD  MOVEMENTS 


SUM  OF  ALL  HM 


( CORRECTED 

FOR  #  FLIGHTS) 

RANKING  OF  HM 

FREQUENCY 

INGRESS 

REFUEL 

INGRESS 

REFUEL 

HMD 

noHMD 

HMD 

noHMD 

HMD 

noHMD 

HMD 

noHMD 

36 

125 

95 

220 

4 

1 

1 

1 

46 

116 

94 

214 

3 

2 

2 

2 

57 

40 

36 

40 

2 

5 

4 

4 

19 

60 

11 

15 

5 

4 

5 

5 

98 

100 

76 

52 

1 

3 

3 

3 
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5.0  Summary 


The  above  analyses  demonstrated  that  consistent  differences 
exist  between  the  HMD  and  noHMD  conditions  associated  with 
the  INGRESS  missions:  pilots  engaged  in  a  greater  percentage 
of  rightward  head  movements  in  the  INGRESS/noHMD  condition. 
Although  the  availability  of  the  HMD  had  no  effect  on  gaze 
shifts  to  the  RIGHT  and  DOWN  (the  location  of  the  map), 
marked  individual  differences  did  exist  with  respect  to  such 
gaze  shifts. 

Additionally,  the  patterns  of  head  movements  associated  with 
the  REFUEL  missions  were  markedly  different  from  those 
associated  with  the  INGRESS  missions,  in  that  there  were  more 
LEFT  head  movements,  and  fewer  RIGHT  and  DOWN  movements. 
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F.  HEAD  AND  EYE  MOVEMENTS 

1.0  Quantitative  Analysis;  Horizontal  Plane 

A  numbe.  of  patterns  of  horizontal  eye  movements  in  conjunc¬ 
tion  with  head  movements  in  the  horizontal  plane  could  be 
identified.  The  most  common  pattern,  and  one  frequently 
referred  to  in  the  literature,  involves  a  saccade  followed  by 
a  slower  eye  movement,  a  "compensatory  eye  movement,"  in  the 
opposite  direction.  It  is  presumed  that  the  saccade  in  con¬ 
junction  with  the  early  phase  of  the  head  movement  allows  the 
observer  to  foveate  the  object.  The  slower  backward  (compen¬ 
satory)  movement  of  the  eye  is  presumed  to  occur  concurrent 
with  a  continued  head  movement,  and  allows  the  viewer  to 
maintain  the  object  in  foveal  view.  These  observations  have 
generally  been  made  in  laboratory  situations  where  the 
location  of  the  item  to  be  fixated  is  under  experimenter 
control,  and  where  the  movement  is  restricted  to  the 
horizontal  plane. 

Our  analysis  of  what  subjects  did  under  our  flight  simulation 
condition  is  considerably  at  variance  with  the  above 
description.  For  subject  #5,  we  sampled  25  horizontal 
eye-head  movements  where  the  eye  movement  involved  a  saccade 
followed  by  a  slower  compensatory  eye  movement  in  the 
direction  opposite  the  saccade.  Approximately  an  equal 
number  of  left-  and  right-going  saccades  and  head  movements 
were  analyzed.  We  identified  three  points  for  each  eye 
movement,  and  two  points  for  each  head  movement.  The  three 
time  points  for  each  saccade  were:  initiation,  saccade 
termination,  and  "compensatory  eye  movement"  termination. 

The  two  head  movement  points  identified  were  head  movement 
initiation  and  termination.  From  these  values  were  derived: 
median  saccade  duration  (180  ms),  compensatory  eye  movement 
duration  (220  ms),  and  head  movement  duration  (370  ms). 
Further,  the  median  delay  of  head  movement  initiation, 
following  saccade  initiation,  was  110  ms,  and  finally,  the 
median  time  between  compensatory  eye  movement  and  head 
movement  termination  was  100  ms. 

Since  saccade  amplitude  and  head  movement  amplitude  were  not 
measured,  amplitude  cannot  be  related  to  duration.  However, 
saccade  duration  medians  of  180  ms  are  not  unusual  for  large 
amplitude  saccades,  which  we  assume  these  were.  Head 
movement  duration  is  quite  independent  of  head  movement 
amplitude,  and  the  ourations  obtained  here  were  within  the 
range  normally  reported. 

The  time  between  saccade  and  head  movement  initiation  found 
under  strict  experimental  conditions  is  on  the  order  of  50 
ms.  Our  value  of  110  ms  is  well  outside  that  range,  but  not 
unusual  for  studies  involving  somewhat  more  complex  visual 
information  acquisition  tasks  than  those  usually  found  in 
laboratory  studies, 
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We  were  quite  surprised,  however,  that  unlike  results 
reported  in  the  literature,  our  compensatory  eye  movements 
did  not  terminate  concurrently  with  head  movement  termina¬ 
tion.  The  eye  movements  frequently  terminated  considerably 
before  head  movement  termination  (100  ms).  Evaluating  the 
time  difference  between  head  movement  initiation  and  compen¬ 
satory  eye  movement  termination,  we  find  the  median  head 
movement  to  be  initiated  approximately  60  ms  before  saccade 
termination.  Thus,  the  compensatory  eye  movements  start  60 
ms  into  the  head  movement  and  may  terminate  long  before  the 
head  movements  ends. 

This  raises  a  major  question  for  us  concerning  the  issue  of 
whether  the  backward  drift  of  the  eye  is  a  compensatory  eye 
movement  made  to  maintain  fixation  on  the  object.  Of  course, 
we  have  not  ruled  out  possible  errors  to  account  for  this 
effect,  and  believe  that  this  phenomenon  needs  to  be  studied 
under  better  controlled  conditions  than  those  available  in  a 
flight  simulator. 

A  second  eye  movement  pattern  seen  in  conjunction  with  head 
movements  in  the  horizontal  plane  involved  a  saccade, 
followed  by  a  slower  eye  movement  (compensatory  eye  movement) 
in  the  same  direction  as  the  saccade,  followed  by  a 
compensatory  eye  movement  in  the  opposite  direction.  This 
pattern  occurs  less  frequently  than  the  one  described 
earlier.  When  this  pattern  occurs,  the  head  movement  is  most 
likely  to  be  initiated  during  the  second  component,  i.e., 
during  the  slow  eye  movement  that  continues  in  the  direction 
of  the  immediately  preceding  saccade.  This  pattern  appears 
to  be  less  frequent  than  the  first  pattern,  occurring  about 
one  third  as  often.  In  this  pattern,  most  of  the  head 
movements  are  initiated  during  the  slower  eye  movement  (the 
one  which  follows  the  saccade).  Saccade  durations  are 
consistently  shorter  in  this  pattern  than  in  the  first 
pattern,  but  the  total  time  of  the  saccade  and  compensatory 
eye  movement  is  longer  than  the  saccade. 

It  is  our  impression  that  we  can  reliably  predict  that  a  head 
movement  is  present  when  either  of  these  two  patterns  of 
horizontal  eye  movement  are  seen.  In  the  vertical  plane,  we 
are  considerably  less  secure  about  inferring  head  movements 
from  the  pattern  of  eye  movements. 

2.0  Qualitative  Analyses,  or,  a  Picture  is  Worth  a  Thousand  Words 
2.1  Computer  Displays 

Figures  23A  to  23S  are  selected  10  second  segments  of 
computer-displayed  data  that  contain  both  head  and  eye 
movements.  The  segments  are  from  the  flights  of  two  sub¬ 
jects  and  were  chosen  to  represent  a  variety  of  patterns 
of  head  and  eye  movements.  No  attempt  at  calibrating  eye 
and  head  movement  amplitudes  was  made;  however,  the  same 
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gain  settings  were  used  for  all  data  sets. 

2.2  Sampling,  Filtering,  and  Digitizing  Procedures 

All  data  were  sampled  at  10  msec  intervals  and  were 
low-pass  (analog)  filtered  (100  Hz  cutoff).  No  other 
filtering  was  done.  It  should  be  pointed  out  that  during 
recording,  the  head  movement  signals  are  digitized  at  50 
msec  sampling  intervals.  The  data  were  then  D/A  converted 
off-line  and  recorded  on  analog  tape,  together  with  the 
EOG  data.  The  fact  that  the  head  movements  were  initially 
sampled  at  50  msec  intervals  is  readily  apparent  in  those 
tracings  with  large  amplitude  head  movements  in  that  the 
intervals  between  successive  steps  are  approximately  50 
msec  in  duration. 

2.3  Display  Format 

In  all  displays,  Channel  1  contains  vertical  EOG.  Blinks 
are  identified  as  downward  going.  From  an  EOG  standpoint, 
an  upward  eye  movement  is  the  same  as  the  lid  moving  down 
over  the  eye  (a  blink).  Thus,  upward  saccades,  as  well  as 
blinks,  are  translated  into  downward  deflections.  Channel 
2  presents  horizontal  EOG  data;  left  is  up  and  right  is 
down.  Channels  3  and  4  present  vertical  and  horizontal 
head  movements.  In  the  vertical  channel,  up  is  up  and 
down  is  down.  In  the  horizontal  plane,  head  left  is  an 
upward  deflection;  head  right  is  downward.  It  should  be 
noted  that  we  have  only  abstracted  part  of  the  data 
available  from  the  Polhemus  head  tracker.  Based  on  our 
observations  of  the  pilots,  we  assume  that  these  two 
dimensions  captured  the  majority  of  head  movement  shifts. 

2.4  Interpretation  of  Displays 

The  data  in  Figure  23A  were  abstracted  from  subject  #l's 
refuel  flight  D,  run  under  HMD  conditions.  They  depict 
the  period  preceding  the  sighting  of  the  tanker  (about  30 
seconds  before  "tally-ho").  The  vertical  lines  drawn  on 
the  tracings  index  the  onset  and  termination  of  head 
movements.  It  should  be  pointed  out  that  when  we  speak  of 
eye  movements,  the  reference  is  a  change  of  eye  position 
with  respect  to  the  head. 

Approximately  1.6  seconds  into  the  display,  the  pilot 
looks  downward  (channel  1)  and  returns  the  eye  to  the 
initial  position  a  half  second  later  ( ch  1).  About  120  ms 
after  the  initiation  of  the  eye  movement,  a  downward  head 
movement  ( ch  3)  is  initiated.  The  return  head  movement 
( ch  3)  is  made  after  the  eye  returns  ( ch  1)  to  the  origi¬ 
nal  fixation  point.  Approximately  8.2  seconds  into  the 
display,  he  looks  both  downward  ( ch  1)  and  to  the  left 
( ch  2).  His  head  movements  down  (ch  3)  and  to  the  left 
( ch  4)  again  lag  the  onset  of  the  eye  movements.  We  can 
also  see  that  the  head  movements  are  accompanied  by  com- 
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pensating  eye  movements  indicating  the  following 
sequence:  his  eyes  saccade  down  and  to  the  left;  his  eyes 
remain  fixated  on  the  same  point  and  his  head  then 
follows  the  eye  movement  (note  the  compensating 
deflections  in  the  eye  movement  channels);  the  amplitude 
of  the  head  movement  is  less  than  that  of  the  eye  move¬ 
ment,  a  fact  that  is  suggested  by  the  raised  channel  1 
and  2  baselines;  finally,  since  no  further  movements 
occur  in  this  block,  the  pilot  is  in  position  in  which 
his  eyes  are  down  and  to  the  left,  fixated  on  the 
peripheral  stimulus,  and  his  head  is  turned  slightly  in 
the  same  direction. 

This  pilot  does  a  lot  of  blinking  (which  are  numbered  in 
tracing  23A).  We  find  that  a  blink  is  initiated  approxi¬ 
mately  300  ms  before  the  head  starts  to  move  downward. 

The  total  duration  of  the  blink  is  200  ms,  with  eye 
position  higher  in  the  visual  field  following  the  blink 
than  at  blink  initiation.  There  is  a  brief  fixation  pause 
(100  ms)  followed  by  the  large  amplitude  downward 
saccade,  descibed  above.  Saccade  duration  is  approximate¬ 
ly  120  ms.  A  compensatory  eye  movement  in  the  direction 
opposite  the  head  movement  is  initiated  concurrently  with 
the  head  movement.  Approximately  150  msec  after  head 
movement  initiation,  we  see  a  small  amplitude  saccade  in 
the  direction  of  the  initial  large  saccade,  followed  by 
stabilization  of  the  eye  before  the  head  movement  is 
completed.  Approximately  500  msec  following  major  saccade 
termination,  we  observed  a  vertical  saccade  coupled  with 
an  eye  blink.  The  saccade/blink  is  initiated  approximate¬ 
ly  100  msec  before  the  head  starts  its  return  movement. 
The  head  movement  is  initiated  at  the  trough  of  the  eye 
blink.  Eye  position  stabilizes  approximately  200  msec 
before  the  head  movement  terminates.  There  are  no 
horizontal  eye  movements  associated  with  this  vertical 
head  and  eye  movement  pattern. 

An  entire  sequence  of  eye  and  head  movements  downward  and 
to  the  left,  and  return  to  the  central  position,  is 
depicted  in  the  next  figure.  The  return  of  the  head  to 
the  central  location  has  the  vertical  component  initiated 
approximately  150  msec  before  the  horizontal  component, 
with  the  two  terminating  concurrently.  Again,  the 
vertical  head  movement  is  initiated  during  a 
bl ink/saccade ,  with  the  eye  stabilized  in  the  vertical 
plane  at  the  point  the  head  movement  starts.  A  second 
blink  occurs  concurrently  with  head  movement  terminj'  .  . 

In  the  horizontal  plane,  a  saccade/gl  i  ssade  is  initi.-ced 
concurrently  with  blink  initiation  in  the  vertical  plane. 
This  eye  movement  is  terminated  with  vertical  head 
movement  initiation.  A  "compensatory"  slow  drift  toward 
the  left  accompanies  the  head  movement  to  the  right,  but, 
again,  is  completed  some  150  ms  before  completion  of  the 
head  movement. 
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what  inferences,  which  may  be  tested  on  other  data,  are 
we  willing  to  make  from  these  two  figures?  In  Figure  23B, 
we  see  that  the  complex  head  movement  has  blinks 
associated  with  termination,  but  not  the  initiation,  of 
the  gaze  shift.  Based  on  other  data,  we  are  willing  to 
make  the  inference  that  acquisition  of  the  peripheral 
information  was  important  to  the  pilot.  A  blink  in 
association  with  gaze  shift  initiation  slows  down  the 
speed  at  which  such  information  becomes  accessible  to  the 
viewer.  A  second  inference  to  be  drawn  from  Figure  23B 
is  that  head  movements  for  picking  up  peripheral 
information  are  most  likely  to  occur  where  gaze  on  the 
peripheral  information  is  long.  "Long,"  in  the  present 
context,  is  about  1.5  seconds.  Approximately  800  ms 
following  completion  of  the  gaze  return,  we  see  an  eye 
movement  which,  in  the  horizontal  plane,  is  approximately 
the  same  amplitude  as  the  eye  movement  associated  with 
the  gaze  shift  involving  a  head  movement.  The  fixation 
duration  here  is  approximately  200  ms. 

It  should  be  noted  that  this  saccade  is  made  during  the 
terminal  portion  of  an  eye  blink,  with  a  vertical  eye 
position  shift  also  occurring  during  the  blink.  One  might 
infer  here  that  acquisition  of  the  peripheral  information 
may  have  served  only  as  a  check  on  the  information 
acquired  during  the  gaze  shift  (involving  a  head 
movement),  since  the  saccade  is  made  during  a  blink. 

Also  note  that  the  horizontal  saccade  made  during  a  blink 
has  the  same  velocity  characteristics  (slope)  as  the 
return  saccade  made  in  the  absence  of  a  blink. 

Figure  23C  depicts  an  example  where,  instead  of 
compensatory  eye  movements,  one  sees  a  head  movement 
that  overshoots  the  target  and  slowly  drifts  backward  to 
a  position  that  is  then  maintained  for  more  than  four 
seconds  before  the  head  returns  to  a  central  position. 
Other  examples  of  complex  head  movements  can  be  seen  in 
Figures  23E,  G,  and  H).  Notice  in  Figure  23C  that  again, 
the  horizontal  eye  movement,  as  well  as  the  initiation  of 
the  compensatory  eye  movement,  precedes  the  initiation  of 
the  head  movement,  and  that  the  compensatory  eye  movement 
terminates  long  before  the  head  movement  is  completed 
(350  msec).  Also,  note  that  the  return  head  movement, 
although  considerably  larger  in  amplitude  than  that  seen 
in  Figure  23B,  does  not  have  the  "characteristic" 
saccade,  glissade,  compensatory  eye  movement  seen  with 
the  previous  head  movements.  This  attests  to  the 
impression  of  the  richness  of  combinations  of  head  and 
eye  movements  that  may  occur.  The  first  seven  figures 
(23a  to  23G)  depict  data  for  subject  1;  the  remaining 
ones  are  for  subject  5,  who  also  does  a  lot  of  blinking. 
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2.5  Conclusions 


The  eye  movement  patterns  associated  with  head  movements 
recorded  in  flight  simulation  are  somewhat  at  variance 
with  the  literature.  A  number  of  reasons  for  the 
discrepancies  can  be  identified.  We  believe  the  most 
important  are,  first,  that  the  eye  and  head  movements 
made  in  the  present  context  were  self-initiated,  rather 
than  triggered  by  stimuli  presented  by  an  experimenter. 
Second,  the  head  movements  in  the  simulator  generally 
involved  both  horizontal  and  vertical  components,  but,  in 
any  case,  were  not  predictable,  while  those  described  in 
the  literature  deal  predominantly  with  eye  movements  to 
acquire  stimuli  presented  in  one  plane  only,  generally 
horizontal.  A  third  reason  may  be  the  complexity  of  the 
display.  In  laboratory  studies,  gaze  has  to  be  shifted 
from  one  location  to  another  with  no  information  avail¬ 
able  between.  The  simulator  environment  is  considerably 
richer;  information  is  presented  between  primary 
locations  that  may  be  of  incidental  utility  to  the  pilot 
and  he  can  acquire  this  information  while  gaze  is 
shifting  between  locations.  This  type  of  information 
acquisition  pattern  has  not  been  studied  in  the 
laboratory,  to  the  best  of  our  knowledge.  A  fourth  reason 
may  be  equipment  artifact.  However,  although  we  had  no 
way  to  determine  the  lag  in  the  movement  recording 
produced  by  the  A/D  and  subsequent  D/A  conversions,  our 
best  guess  is  that  this  lag  is  of  the  order  of  milli¬ 
seconds,  at  most. 

In  all  data  depicted,  head  movements  lagged  eye  movement 
initiations,  in  both  the  vertical  and  horizontal  planes, 
by  somewhere  between  100  and  200  ms.  This  is  considerably 
longer  than  the  50  msec  lag  that  we  and  others  (Bizzi, 
1974)  have  observed  in  both  monkey  and  man,  where  task 
demands  are  relatively  simple.  There  are  others,  of 
course,  who  have  reported  latencies  in  the  range  obtained 
here  (Bartz,  1966;  Calhoun  et  a]^.  ,  1986;  Gresty,  1974  ; 
Mourant  &  Crimson,  1977).  This  leads  us  to  believe  that 
these  lags  are  not  equipment  artifacts,  but  valid 
measures . 

The  most  interesting  (and  difficult  to  rationalize) 
result  is  that  the  so-called  compensatory  eye  movements 
(compensating  for  head  movements)  are  frequently  found  to 
be  initiated  in  anticipation  of  head  movement,  and 
terminate  before  the  head  finishes  moving.  As  such, 
therefore,  they  must  be  centrally  controlled,  rather  than 
simply  mechanical  reactions  to  head  movements  under 
conditions  where  the  viewer  wishes  to  maintain  tne 
peripheral  object  in  foveal  vision. 

Both  the  glissade  at  the  end  of  a  saccade  and  the 
compensatory  eye  movements  are  relatively  slow  movements. 
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and  therefore  do  not  suffer  from  the  visual  suppression 
associated  with  the  more  rapid  saccadic  eye  movement. 
Thus,  to  the  extent  that  the  viewer  can  utilize  these 
types  of  eye  movements  instead  of  saccades,  he  can  "see" 
environmental  stimuli  earlier  than  would  be  true  under 
saccade  only  conditions. 
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H.  GENERAL  SUMMARY 


1.0  Photic  Stimulation 


Reasonable  evidence  for  photic  driving  could  be  found  in  the 
simulator  data.  Although  some  evidence  suggested  that  the 
degree  of  photic  driving  was  a  function  of  "momentary" 
workload,  this  finding  was  not  consistently  manifested  in  the 
details  of  the  data.  Reasons  for  this  are  proposed  and 
generally  indicate  the  necessity  for  research  that  is  more 
tightly  controlled  than  is  possible  in  the  simulator.  This 
research  (described  in  the  report)  would  be  designed  to 
determine  the  effect  of  a  variety  of  parameters  on  the 
magnitude  of  steady-state  EPs,  such  as: 

a)  Intensity  (and  its  relation  to  critical  flicker 
frequency ) 

b)  Consistent  and  quantifiable  workloads 

c)  Workload  quality 

d)  Procedures  to  maintain  the  constancy  of  the  visual 
input  to  subjects  whose  gaze  is  shifting. 

2 . 0  Blinks  and  Saccades 

Differences  between  the  HMD  and  noHMD  conditions  were  obtained, 
suggestive  of  higher  workload  under  NoHMD  conditions.  These 
differs,  rs  were  found  for  pilot  activity  associated  with 
waypointj  in  the  ingress  flight  condition,  and  for  all  segments 
of  tanker  rendezvous.  No  differences  were  found  during  threat 
avoidance  and  post-threat  conditions. 

This  HMD/NoHMD  difference  was  reflected  most  strongly  in  the 
degree  of  conjoint  occurrence  of  eye  blinks  and  saccades. 
Earlier  work  suggested  that  the  incidence  of  concurrent  blinks 
and  saccades  increases  under  conditions  of  high  visual  task 
demand.  Since  the  present  data  indicate  that  the  bl ink/saccade 
coincidence  is  greacer  under  the  noHMD  condition,  we  infer  that 
use  of  the  helmet  display  reduces  visual  task  demands.  A  second 
measure  (and  only  slightly  less  discriminative  of  workload 
demands)  was  saccade  frequency;  more  visual  searching  is  done 
under  the  noHMD  than  the  HMD  condition.  These  results  suggest: 

a)  Pilots  rapidly  learn  to  make  use  of  the  information 
presented  on  the  HMD  in  favor  of  obtaining  such 
information  from  the  instrument  panel,  but  that 

b)  They  do  not  use  the  added  time  available,  as  a 
consequence  of  this  more  efficient  information 
abstracting  capability,  in  scanning  their  environment 
(presuming  such  scanning  is  desirable). 

c Specific  training  or  further  practice  may  be  necessary 
to  assure  better  utilization  of  "released  time." 

3 . 0  Head  Movements  and  Saccades 

Patterns  of  head  and  eye  movements  observed  in  the  flight 
simulator  are  considerably  at  variance  with  most  data  collected 
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under  more  highly  controlled  laboratory  conditions,  i.e.,  our 
ability  to  generalize  information  collected  under  laboratory 
conditions  may  be  poorer  than  expected.  Some  discrepancies 
between  simulator  and  laboratory  data  are: 

a)  The  timing  between  saccade  and  head  movement  initiation 
was  consistently  longer  under  flight  simulation  than 
laboratory  conditions. 

b)  Eye  movement  patterns  were  more  complex  than  seen  under 
laboratory  conditions. 

c)  Compensatory  eye  movements  frequently  terminated  before 
the  end  of  a  head  movement. 

A  number  of  reasons  for  the  lack  of  generalizability  can  be 
hypothesized.  Most  important  among  these  are: 

a)  In  the  simulator,  gaze  shifts  are  self-directed,  while 
in  the  laboratory,  they  are  initiated  at  experimenter- 
controlled  time  points. 

b)  Information  content  of  the  "display"  in  the  simulator 
is  considerably  richer  and  variable  (dynamic)  than  in 
the  laboratory. 

These  results  suggest  that  the  dynamic  relationship  between 
eye  and  head  movement  is  more  likely  to  be  a  function  of 
central  information  processing  demands  than  of  more  purely 
perceptual  variables. 
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I .  APPENDIX  A 


B-IB  REPRESENTATION 

1 . 0  Description  of  the  F-15E  Cockpit 

An  F-15E  simulator  will  be  used  for  the  B-lB  HMD  evaluation. 
Some  changes  have  been  made  to  its  software  so  as  to  imitate 
the  B-1 .  The  cockpit  layout,  shown  in  Figure  A-1,  has  not 
been  changed.  The  following  description  includes  only  those 
displays  and  instruments  that  will  be  used  during  the  test. 

The  F-15E  has  a  "glass  cockpit."  It  features  three  cathode 
ray  tubes  (CRT),  multipurpose  displays  (MPD),  an  integrated 
comm/nav  control  panel,  which  we  refer  to  as  the  "Up  Front 
Control"  (UFC),  and  a  wide  field  of  view  (WFOV)  head  up 
display  (HUD).  This  cockpit  is  designed  for  maximum  flexi¬ 
bility;  it  is  possible  to  place  a  variety  of  displays  on  each 
of  the  MPDs,  including  radar  imagery,  infrared  imagery, 
weapon  video,  and  systems  status.  Only  the  left  and  right 
MPDs  will  be  used  for  the  B-1  test. 

The  HUD  will  be  present  but  will  not  be  used  to  display 
information.  A  set  of  "round  dial"  backup  flight  instruments 
are  located  on  the  left  subpanel  below  the  left  MPD.  These 
instruments  will  be  the  only  source  of  calibrated  airspeed 
and  mean  sea  level  (MSL)  altitude  when  the  HMD  is  off. 

On  the  right  subpanel,  below  the  MPD,  there  is  an  integrated 
engine  display  and  an  integrated  fuel  display.  The  RPM 
indications  on  the  engine  display  will  be  functional  during 
the  B-1  test,  but  will  not  be  calibrated  to  match  B-1  engine 
settings.  They  are  intended  merely  to  provide  some  indica¬ 
tion  of  thrust  setting  to  the  pilot. 

The  F-15E  stick  grip  and  throttle  grips,  shown  in  Figure  A-2, 
incorporate  numerous  switches.  We  refer  to  these  as  "HOTAS" 
switches  (Hands  On  Throttle  And  Stick).  In  the  F-15,  they  are 
used  to  control  aircraft  sensors  and  weapons  functions.  Only 
five  of  these  switches  will  be  active  during  the  B-1  test. 
Their  use  is  explained  below. 

2 . 0  Genair  B-lB  Flight  Model 

Genair  is  a  generic  flight  model  enabling  the  modeling  of  the 
handling  characteristics  of  any  aircraft.  In  this  program, 
it  is  used  to  simulate  the  B-l's  flight  handling,  and  has 
been  judged  by  two  B-1  pilots  as  being  very  faithful  in  this 
respect  for  the  two  mission  segments  to  be  used:  tanker 
rendezvous  and  low  level  threat  penetration. 

3 . 0  Vision  Restriction 

The  F-15E  has  a  two-seat,  tandem  mounted  cockpit  whereas  the 
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Figure  A-1.  F-15E  cockpit 
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B-lB  has  a  side-by-side  cockpit  that  restricts  the  pilot's 
vision  somewhat.  To  perform  a  reasonable  evaluation  of  the 
HMD,  when  used  in  this  type  of  aircraft,  the  B-1  window 
configuration  has  been  emulated  by  placing  a  foam  core  shell 
(weighing  ten  pounds)  over  the  F-15  cockpit  (Figure  A-3).  It 
is  not  fastened  to  the  cockpit;  it  merely  rests  on  the  cock¬ 
pit  canopy  sills.  This  shell  closely  matches  the  aircraft  on 
the  left  side  and  directly  in  front  of  the  pilot  but  the 
simulator  dimensions  preclude  adding  a  right  side  to  the 
cockpit.  As  a  partial  solution,  the  right  wall  of  the  shell 
has  been  designed  to  appear  like  a  B-1  cockpit  to  the  pilot 
when  he  looks  cross-cockpit  from  his  normal  position.  The 
pilot  will  defeat  the  intent  of  this  device  if  he  moves  his 
head  excessively  to  the  right  and  will  be  instructed  to  avoid 
this  movement. 

4 . 0  Cockpi t  Displays 

The  left  and  right  MPDs  will  be  used  during  the  HMD 
evaluation.  The  pilot  should  not  need  to  turn  these  on,  but 
may  want  to  adjust  display  brightness  and/or  constrast.  The 
controls  for  this  are  shown  in  Figure  A-4,  and  will  be 
demonstrated  during  the  training  sessions.  They  are  as 
follows : 

4.1  B-1  VSD  Display  -  The  left  MPD  will  be  used  to  provide  a 
simulated  B-1  VSD,  shown  in  Figure  A-5.  This  should 
behave  similarly  to  the  aircraft  VSD,  but  a  few  comments 
are  in  order  to  describe  how  it  has  been  programmed: 

4.1.1.  Range  Indicator  -  The  range  numeric  in  the  lower 
left  corner  will  indicate  range  to  the  tanker  during 
the  tanker  rendezvous  missions,  and  range  to  the  next 
waypoint  during  low  level  penetration  missions. 

4.1.2.  Ground  Speed  Indicator  -  This  numeric  always  shows 
ground  speed. 

4.1.3.  Altitude  -  MSL  altitude  will  be  displayed  at  all 
times  in  the  upper  right  corner  of  the  VSD.  This 
will  be  provided  only  for  altitudes  below  5000  ft. 

4.1.4  Steering  Cross  -  During  tanker  rendezvous 

missions,  the  steering  cross  will  be  caged  at  the 
center  of  the  aircraft  symbol.  During  low  level 
penetration,  the  steering  cross  will  provide  command 
steering  to  the  centerline  of  each  leg  of  the  planned 
route . 

4.2  Wing  Sweep  Indicator  -  The  right  MPD  will  be  used  to 
provide  a  Wing  Sweep  Indicator,  shown  in  Figure  A-6.  The 
top  of  the  scale  represents  the  forward  sweep  limit  of 
15°;  the  bottom  represents  the  aft  limit  of  67.5°.  Each 
tick  mark  on  the  scale  represents  a  10°  increment.  The 
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Figure  A-^.  Multipurpose  display  controls 
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ULATOR  VSD  FORMAT 
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Figure  A-6.  Simulation  wing  sweep  indicator 
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carat  and  number  on  the  left  of  the  scale  indicates 
present  wing  position. 

The  bar  on  the  right  of  the  scale  indicates  the  forward 
and  aft  wing  limits  based  on  current  flight  conditions. 
This  is  depicted  in  Figure  A-7. 

The  allowable  range  bar  is  dynamic  and  is  continually 
repositioned  to  depict  the  present  wing  position  limits 
as  a  function  of  flight  conditions.  If  changing  flight 
conditions  result  in  the  present  position  being  out  of 
limits,  the  wings  move  as  necessary  to  remain  at  either 
the  forward  or  aft  limit  (depending  on  which  is 
critical).  The  wings  will  move  to  the  position  selected 
by  the  pilot  whenever  this  position  is  within  the 
allowable  range.  The  pilot  selects  wing  position  with 
the  wing  sweep  switch  on  the  throttle. 

B.  B-1  HELMET  MOUNTED  DISPLAY 

1.0  Kaiser  Agile  Eye  System 

The  Agile  Eye  helmet  mounted  display  (HMD)  system  is 
manufactured  by  Kaiser  Electronics.  A  prototype  version  of 
this  system  will  be  used  for  this  test.  Because  this  is  a 
prototype,  some  aspects  of  the  system  are  less  than  ideal. 

The  HMD  system  is  composed  of  three  major  subsystems:  the 
helmet  itself,  the  head  tracker  and  the  display  system. 

These  are  illustrated  in  Figure  A-8  and  described  briefly 
below. 

Agile  Eye  Helmet  -  The  Agile  Eye  helmet  was  designed  as  a 
new  flight  helmet  with  the  HMD  integrated  into  it  from 
the  beginning.  The  new  design  improved  three  basic 
characteristics  of  the  present  flight  helmet.  The  amount 
of  aerodynamic  lift  at  high  speed  is  cut  in  half.  The 
weight  was  redistributed  so  that  the  overall  CG  of  this 
helmet  is  closer  to  the  natural  CG  of  the  human  head  than 
present  helmets.  The  total  weight,  with  tracker  and  dis¬ 
play  elements,  is  one-half  pound  lighter.  This  is  done  by 
using  new  components  and  Kevlar  for  the  shell,  which 
provides  the  strength  required  for  a  flight  helmet,  but 
greatly  reduces  the  weight. 

1.2  Head  Tracker  -  The  Agile  Eye  system  employs  an 

electromagnetic  tracking  system  manufactured  by  Polhemus 
Navigation  Sciences.  A  source,  positioned  above  the 
pilot's  head,  radiates  three  orthogonal  magnetic  signals. 
In  the  simulator,  the  source  is  mounted  on  an  arm  above 
and  attached  to  the  ejection  seat.  In  an  aircraft,  it  can 
be  mounted  anywhere,  as  long  as  it  has  a  clear  line  of 
sight  to  the  helmet. 
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The  receiver  is  mounted  on  the  inside  and  at  the  top  of 
the  Agile  Eye  helmet.  Changes  in  the  orientation  of  the 
helmet  relative  to  the  source  cause  changes  in  the 
received  signal.  By  comparing  the  two,  helmet  position 
and  orientation  are  determined.  From  these,  line  of  sight 
(LOS)  angles  are  computed.  The  display  is  then  tailored 
for  the  LOS.  Accuracy  is  1/2  degree,  which  is  adequate 
for  most  flight  operations. 

1.3  Helmet  Display  System  -  The  display  is  provided  by  a 

half-inch  diameter  cathode  ray  tube  (CRT)  mounted  inside 
the  helmet  on  the  right  side.  The  CRT  image  is  carried 
to  the  helmet  visor  optically,  using  fiber  optics  and 
mirrors.  The  visor  is  the  final  optical  element  and  acts 
as  the  combining  glass  for  the  display,  superimposing  the 
display  elements  on  the  outside  visual  scene.  This 
system  produces  a  display  that  has  a  12°  field  of  view 
(FOV),  which  can  be  positioned  at  any  LOS  that  the  pilot 
can  achieve  through  head  movements. 

The  video  signal  for  the  display  is  generated  by  a 
digital  graphics  processor.  This  signal  is  transmitted 
to  the  helmet  at  a  low  voltage  level  so  that  no  spark  is 
produced  by  quick  disconnect  of  the  necessary  electrical 
lines.  A  high  voltage  power  supply  is  located  within  the 
helmet,  and  converts  the  input  video  signal  to  the 
voltage  levels  necessary  to  drive  the  helmet  CRT. 

2.0  HMD  Controls 

2.1  HMD  Control  Panel  -  The  control  panel  for  the  HMD  is 
shown  in  Figure  A-9 .  The  only  control  necessary  to 
activate  on  this  panel  is  the  HMD  brightness  control, 
which  should  be  adjusted  as  desired  at  the  start  of  each 
session.  This  panel  is  located  on  the  left  console. 

2.2  HOTAS  Swi tches  -  Two  HOTAS  switches  are  used  to  operate 
the  HMD.  Use  of  the  boresight  switch,  located  on  the 
stick,  is  covered  in  the  next  paragraph.  The  blanking 
switch,  located  on  the  left  throttle,  is  used  to 
temporarily  turn  the  display  off  by  holding  the  switch 
depressed.  The  location  of  these  switches  is  shown  in 
Figure  A-10 . 

2.3  Helmet  Boresight  Procedure  -  The  position  and  orientation 
of  the  helmet  can  be  determined  very  precisely.  However, 
a  known  reference  must  first  be  supplied  to  the  system. 
This  is  accomplished  by  boresighting  the  helmet.  We  have 
provided  a  very  simple  mechanism  for  accomplishing  this. 
At  the  start  of  each  session,  simulation  engineers  will 
place  the  Simulator  in  "Reset"  mode.  To  "boresight  the 
helmet,"  the  Boresight  Switch  on  th^  stick  grip  is 
pressed  and  held.  A  box  with  a  center  cross  will  be 
displayed  on  both  the  F-15  HUD  and  the  Agile  Eye  HMD,  as 
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shown  in  Figure  A-11.  The  pilot's  head  is  positioned  so 
as  to  align  the  two  symbols  visually.  When  the  two  are 
aligned,  the  Boresight  Switch  is  released. 

3.0  Helmet  Mounted  Display  (HMD)  Formats 

3.1  Basic  Display  -  The  basic  format,  which  is  present 
whenever  the  HMD  is  on,  is  shown  in  Figure  A-12.  A 
description  of  display  elements  is  provided  below. 

3.2.1  Heading  Indication  and  Scale  -  The  three  digit 
number  below  the  scale  at  the  top  indicates  aircraft 
magnetic  heading. 

3.2.2  Horizon  Bar  -  The  location  of  the  real  world 
horizon  relative  to  head  orientation  is  indicated  by  the 
horizon  bar.  It  is  always  drawn  parallel  to  the  real 
horizon.  When  the  real  horizon  is  outside  the  HMD  FOV, 
the  horizon  bar  will  be  displayed  as  a  dashed  line  at  the 
edge  of  the  display  FOV,  but  it  will  remain  parallel  to 
the  real  horizon.  This  case  is  shown  in  Figure  A-13. 

The  horizon  bar  employs  ground  pointers  and  slty  pointers 
to  indicate  which  side  of  the  bar  is  down  (toward  the 
ground).  When  the  pilot's  head  is  positioned  within  6“  of 
aircraft  heading,  there  will  be  a  gap  in  the  bar 
indicating  the  direction  in  which  the  aircraft  is  flying. 
The  tick  marks  on  either  side  of  this  gap  are  ground 
pointers,  and  indicate  which  side  of  the  bar  is  the 
ground  side.  As  the  pilot  moves  his  head  to  either  side, 
the  gap  will  slide  to  the  edge  of  the  HMD  FOV,  and  will 
be  removed  from  the  display  once  the  6°  point  is  reached. 
At  this  time,  a  triangle  symbol  will  appear  on  the 
opposite  side  of  the  HMD,  as  shown  in  Figure  A-14.  This 
is  a  sky  pointer,  and  indicates  which  side  of  the  bar  is 
the  sky  side.  As  the  pilot's  head  angle  relative  to  the 
aircraft  nose  increases,  the  triangle  will  move  toward 
the  center  of  the  HMD  until  12°  is  reached.  At  this 
time,  the  triangle  will  be  centered  on  the  horizon  bar, 
and  will  remain  centered  until  the  angle  is  reduced  below 
12°.  Figure  A-14  illustrates  the  case  where  the  pilot's 
head  is  being  moved  to  the  right. 

It  should  be  noted  that  the  horizon  bar  indicates  horizon 
location,  NOT  aircraft  attitude.  However,  the  bar  does 
provide  an  extreme  attitude  warning,  as  discussed  in 
paragraph  3.3  below, 

3.2.3  Waterline  -  The  "Waterline"  symbol  indicates  the 
position  of  the  nose  of  the  aircraft  relative  to  the 
horizon.  This  symbol  is  space  stabilized  to  the  nose 
of  the  aircraft,  and  will  disappear  off  the  display 
as  you  move  your  head  away  from  the  nose. 
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Figure  A-12.  Basic  B-1  HMD  format 
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Haad  2:  12"  OH  Nose 


Figure  A-14.  [Ielket  pointed  av/ay  from  nose  of  aircraft 


3.2.4  Velocity  Vector  -  The  velocity  vector  indicates 
the  point  towards  which  the  aircraft  is  flying 
(actual  flight  path).  Thus,  if  the  aircraft  is  flown 
so  as  to  place  the  velocity  vector  on  the  horizon 
bar,  the  aircraft  will  be  in  level  flight, 
maintaining  a  constant  altitude.  This  will  be  true 
regardless  of  airspeed,  angle-of-attack ,  or  aircraft 
attitude.  Note  that  the  velocity  vector  normally 
does  not  coincide  with  aircraft  nose  position.  The 
waterline  symbol  provides  aircraft  nose  position,  and 
it  is  usually  above  the  velocity  vector,  reflecting 
positive  angle-of-attack.  The  velocity  vector 
indicates  aircraft  flight  path  as  opposed  to  aircraft 
pitch  attitude. 

3.2.5  Ai r speed  -  Airspeed  is  displayed,  to  the  nearest 
knot,  on  the  left  at  the  bottom  of  the  display.  It 
is  distinguished  from  altitude  by  the  "K"  symbol 
(knots),  which  follows  the  digits.  All  digits  are 
drawn  full  size  (approximately  5  mils  x  7  mils). 

This  display  will  provide  either  calibrated  airspeed 
(CAS)  or  ground  speed  (GS),  depending  on  the  phase  of 
flight.  During  the  tanker  rendezvous,  CAS  will  be 
displayed.  During  the  low  level  penetration,  GS  will 
be  displayed. 

3.2.6  Mean  Sea  Level  ( MSL )  Altitude  -  MSL  Altitude  is 
displayed  on  the  right  at  the  bottom  of  the  display. 
Thousands  of  feet  are  displayed  as  full  size  digits; 
hundreds  of  feet  are  displayed  by  a  single  digit, 
which  is  75%  of  full  size. 

3.2.7  Bank  Scale  -  Aircraft  bank  angle  is  displayed  by 
the  bank  scale  at  the  bottom  of  the  display.  Tick 
marks  indi .ate  every  10“  of  bank  up  to  30°  bank  left 
or  right.  If  30“  bank  is  exceeded,  the  bank  scale 
pointer  will  move  off  the  scale  and  continue  until 
45“  bank  is  reached.  If  45“  bank  is  exceeded,  the 
bank  pointer  remains  "pegged"  at  45“  flashes. 

3.3  Extreme  Attitude/Ground  Proximity  Wa rning  -  The  system 
provides  a  warning  of  extreme  attitudes  as  a  function  of 
ground  proximity.  The  triggering  conditions  are  set  for 
fighter  aircraft  operations,  so  you  will  probably  not  see 
this  warning.  It  is  provided  whenever  either  set  of  the 
following  conditions  occurs: 

A.  Dive  Angle  between  30°  and  60°,  and  altitude  at  or 
below  5000  ft.  MSL. 

B.  Dive  Angle  60“  or  greater  and  altitude  at  or  below 
10,000  ft  MSL. 

When  triggered,  the  horizon  bar  flashes  at  2.5  Hz. 

These  altitude/dive  angle  combinations  provide  sufficient 
altitude  for  dive  recovery  with  a  fighter.  No  attempt  has 
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been  made  to  convert  them  to  B-1  performance. 

3.4  Ai r  Target  Symbols  -  When  another  aircraft  is  being 
tracked,  one  of  two  symbols  will  be  present  on  the  HMD: 
the  Target  Designator  (TD),  a  box  positioned  at  the 
target's  line  of  sight  (LOS);  or  the  Target  Pointer  (TP), 
which  is  an  arrow  pointing  toward  the  target  LOS.  Both 
symbols  are  illustrated  in  Figure  A-15. 

If  the  target  is  not  located  within  the  HMD  FOV,  then  the 
TP  arrow  will  be  present.  The  TP  arrow  radiates  out  from 
the  center  of  the  HMD  FOV  at  an  angle  which  indicates  the 
shortest  direction  to  the  target  LOS.  By  moving  his  head 
in  the  direction  indicated,  the  pilot  is  moving  the 
helmet  to  a  position  which  aligns  the  helmet  LOS  to  the 
target  LOS.  This  helps  the  pilot  locate  the  TD  box. 

The  TD  box  is  a  square  that  measures  25  milliradians  (mr) 
on  each  side.  The  length  of  the  TP  arrow  is  fixed  at  50 
mr  until  the  target  is  within  45®  of  the  helmet  LOS. 

Then,  it  shortens  at  a  rate  of  1  mr/degree,  until  the 
target  is  within  the  HMD  FOV  (arrow  is  approximately  11 
mr  long).  Once  the  target  is  within  the  HMD  FOV,  the  TD 
box  appears  around  the  target,  and  the  TP  arrow 
disappears.  The  two  symbols  are  mutually  exclusive; 
there  is  never  a  situation  when  both  are  displayed  at  the 
same  time. 

When  the  TD  box  is  displayed,  closure  rate  and  range  to 
the  tracked  aircraft  is  displayed  by  two  numbers.  The 
number  above  the  TD  box  provides  closing  velocity  in 
knots.  If  the  relative  position  of  the  two  aircraft  (B-1 
and  target)  is  such  that  they  are  moving  apart,  this 
number  is  prefixed  with  a  negative  sign  (-).  The  number 
below  the  TD  box  indicates  range  to  the  target  to  the 
nearest  0.1  NM. 

The  Air  Target  format  will  be  displayed  during  the  tanker 
rendezvous.  It  is  assumed  that  the  operation  of  locating 
and  tracking  the  tanker  is  being  performed  by  aircraft 
systems,  and  these  systems  are  providing  the  data 
necessary  to  drive  the  displays. 

3.5  Ground  Target  Symbols  -  when  performing  navigation  to  a 
ground  point,  two  symbols  will  be  present  on  the  HMD: 

the  waypoint  symbol  and  the  command  heading  marker.  Both 
are  depicted  in  Figure  A-16. 

The  waypoint  symbol  is  a  circle  25  mr  in  diameter  that  is 
positioned  at  the  LOS  of  the  waypoint  on  the  ground.  The 
number  inside  the  circle  indicates  waypoint  number  (e.g., 
waypoint  4).  The  number  below  the  circle  provides 
time-to-go  to  the  waypoint  in  minutes  and  seconds.  When 
the  waypoint  LOS  is  outside  the  FOV  of  the  HMD,  the 
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Figure  A-16.  Ground  target  display 


waypoint  symbol  is  limited  to  indicate  the  direction  to 
look  to  find  the  waypoint.  This  is  shown  in  Figure  A-17. 
The  command  heading  marker  is  a  small  triangle  and 
pointer  positioned  below  the  heading  scale  at  the  top  of 
the  HMD.  When  aircraft  heading  is  within  4° 
(approximately)  of  the  commanded  heading,  the  command 
heading  marker  will  be  positioned  appropriately  on  the 
heading  scale.  If  the  aircraft  heading  is  more  than  4° 
different  from  commanded  heading,  the  command  heading 
marker  will  be  pegged  in  the  direction  the  aircraft 
should  be  turned. 

The  waypoint  symbol  and  command  heading  marker  always 
indicate  the  next  waypoint  in  the  mission.  The  system 
automatically  sequences  to  the  next  waypoint  on  the  route 
when  the  pilot  crosses  a  waypoint.  In  this  simulation, 
all  turns  will  be  made  after  crossing  the  waypoint,  as 
opposed  to  turning  early  to  intercept  a  course 
centerline . 

The  Ground  Target  format  will  be  displayed  during  the  low 
level  penetration  portion  of  each  run.  It  is  assumed 
that  the  OSO  performs  the  necessary  operations  to  supply 
waypoint  location  to  the  aircraft  systems,  and  these 
systems  are  providing  the  data  necessary  to  drive  the 
displays . 
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Figure  A-17.  Ground  target  outside  HHD  FOV 


